-- Logs begin at Wed 2022-10-12 01:51:01 UTC, end at Wed 2022-10-12 02:54:17 UTC. -- Oct 12 02:20:51.725712 dv-cinder-849143-14-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 12 02:20:51.734393 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 12 02:20:51 2022] *** Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: nodename: dv-cinder-849143-14-iscsi Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: machine: x86_64 Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: clock source: unix Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: pcre jit disabled Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: detected number of CPU cores: 4 Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: current working directory: / Oct 12 02:20:51.736947 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: detected binary path: /usr/bin/uwsgi-core Oct 12 02:20:51.737649 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: your processes number limit is 47680 Oct 12 02:20:51.737649 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: your memory page size is 4096 bytes Oct 12 02:20:51.737649 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: detected max file descriptor number: 2048 Oct 12 02:20:51.737649 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: lock engine: pthread robust mutexes Oct 12 02:20:51.737649 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: thunder lock: enabled Oct 12 02:20:51.737649 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 12 02:20:51.737649 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 12 02:20:51.776455 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: Python main interpreter initialized at 0x55a6869cb9c0 Oct 12 02:20:51.776455 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: python threads support enabled Oct 12 02:20:51.776455 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: your server socket listen backlog is limited to 100 connections Oct 12 02:20:51.776455 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: your mercy for graceful operations on workers is 90 seconds Oct 12 02:20:51.776851 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: mapped 403077 bytes (393 KB) for 2 cores Oct 12 02:20:51.776941 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: *** Operational MODE: preforking *** Oct 12 02:20:51.777144 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: *** uWSGI is running in multiple interpreter mode *** Oct 12 02:20:51.777144 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: spawned uWSGI master process (pid: 103652) Oct 12 02:20:51.777316 dv-cinder-849143-14-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 12 02:20:51.778033 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: spawned uWSGI worker 1 (pid: 103653, cores: 1) Oct 12 02:20:51.778243 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: spawned uWSGI worker 2 (pid: 103654, cores: 1) Oct 12 02:20:51.778780 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 12 02:20:52.430584 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 12 02:20:52.432857 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 12 02:20:53.508685 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:20:53.508685 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: from cryptography.utils import int_from_bytes Oct 12 02:20:53.509275 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:20:53.509275 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: from cryptography.utils import int_from_bytes Oct 12 02:20:53.510401 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:20:53.510401 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: from cryptography.utils import int_from_bytes Oct 12 02:20:53.510841 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:20:53.510841 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: from cryptography.utils import int_from_bytes Oct 12 02:20:55.953852 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 02:20:55.954428 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 02:20:55.954615 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 02:20:55.954964 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 02:20:55.955211 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 02:20:55.955590 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.955910 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.956223 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.956535 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.956882 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.957187 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.957349 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 02:20:55.957725 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.957813 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 02:20:55.957906 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.958099 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.958194 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 02:20:55.958350 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.958438 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 02:20:55.958654 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.958755 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 02:20:55.958965 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849143-14-iscsi {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.959059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.959277 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.959363 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.959575 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.959669 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.959874 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.959960 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.960180 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.960273 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.960473 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.960561 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.960810 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.960900 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.961112 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.961204 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.961514 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.961600 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.961778 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.962027 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.962138 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.962325 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.962420 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849143-14-iscsi {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.962622 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.962723 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.962947 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.963041 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.963252 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.963358 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.963551 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.963644 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.963857 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.963961 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.964153 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.964245 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.964466 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.964557 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.964789 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.964969 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.965187 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849143-14-iscsi {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.965288 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.965497 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.965606 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.965801 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.965893 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.966103 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.966203 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.966466 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.966557 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.966771 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.966862 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.967078 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.967169 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.967379 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.967471 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.967691 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.967785 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.967990 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.968098 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.968308 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.968404 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849143-14-iscsi {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.968617 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.968740 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.969003 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.969666 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.969813 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.969972 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.970112 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.970208 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.970414 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.970500 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.970713 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.970808 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.971028 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.971117 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.971326 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.971411 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.971621 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.971715 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.971865 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.972014 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.972165 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.972332 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.972477 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.972646 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.972817 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.972979 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.973133 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.973280 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.973380 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.973588 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.973676 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.973893 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.973978 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.974207 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.974316 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.974504 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.974596 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.974804 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.974904 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.975067 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.975224 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.975390 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.975475 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.975689 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.975782 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.975985 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.976073 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.976284 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.976382 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.976583 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.976668 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.976877 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.976966 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.977194 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.977281 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.977511 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.977608 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.977813 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.977910 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.978115 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.978199 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.978421 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.978525 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.978726 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.978826 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.979025 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.979111 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.979326 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.979423 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.979629 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.979715 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.979931 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.980017 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.980239 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.980331 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.980548 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.980633 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.980926 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.981018 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.981232 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.981331 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.981532 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.981625 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.981842 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.981927 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.982159 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.982252 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.982452 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.982536 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.982695 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.982791 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.982986 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.983074 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.983282 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.983367 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.983590 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.983675 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.983887 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.983974 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.984184 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.984282 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.984483 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.984583 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.984833 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.984975 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.985120 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.985206 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.985419 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.985523 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.985717 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.985804 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986014 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986107 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986312 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986406 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986608 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986703 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986900 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.986995 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.987197 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.987301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.987492 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.987578 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.987786 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.987937 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.988082 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.988176 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.988387 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.988472 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.988683 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.988795 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.989007 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.989095 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.989303 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.989391 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.989601 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.989687 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.989902 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.990064 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.990196 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.990288 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.990493 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.990586 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.990792 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.990877 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.991092 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.991176 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.991384 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.991529 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.991693 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.991788 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.991994 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.992081 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.992294 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.992387 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.992596 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.992683 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.993168 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.993263 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.993470 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.993559 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.993771 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.993857 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.994070 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.994155 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.994367 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.994485 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.994661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.994760 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.994955 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.995094 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.995249 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.995342 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.995544 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.995634 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.995834 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.995921 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.996127 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.996225 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.996417 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.996503 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.996738 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.996827 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.997052 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.997149 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.997351 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.997438 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.997648 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.997731 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.997941 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.998080 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.998238 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.998322 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.998537 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.998623 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:55.998836 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.998932 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:55.999135 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.999220 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:55.999433 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.999518 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:55.999729 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:55.999822 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.000041 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:56.000126 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.000393 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:56.000480 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.000711 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:56.000857 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.001020 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:56.001106 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.001320 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:56.001405 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.001633 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.001719 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:20:56.001938 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.002023 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.002243 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.002339 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.002555 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.002643 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.002869 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.002956 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.003194 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.003291 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.003504 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.003591 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.003811 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.003896 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.004121 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.004216 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.004425 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.004521 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.004758 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.004861 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.005070 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.005165 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.005381 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.005467 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.005686 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.005782 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.006002 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.006101 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.006320 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.006413 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.006632 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.006724 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.006948 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.007045 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.007258 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.007343 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.007561 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.007649 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.007863 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.007954 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.008166 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.008254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.008490 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.008574 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.008818 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.008908 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.009129 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.009216 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.009441 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.009530 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.009747 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.009843 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.010060 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.010146 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.010368 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.010461 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.010670 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.010758 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.010980 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.011065 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.011283 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.011375 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.011583 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.011671 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.011909 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.011998 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.012216 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.012311 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.012526 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.012611 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.012856 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.012946 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.013165 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.013262 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.013475 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.013562 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.013783 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.013869 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.014088 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.014184 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.014394 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.014491 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.014691 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.014776 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.014990 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.015085 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.015289 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.015376 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.015593 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.015728 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.016439 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.016528 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.016778 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.016867 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.017092 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.017176 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.017393 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.017480 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.017704 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.017796 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.018005 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.018098 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.018311 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.018408 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.018614 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.018698 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.018915 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.019003 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.019216 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.019302 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.019521 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.019605 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.019821 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.019917 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.020117 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.020203 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.020433 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.020518 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.020757 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.020863 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.021059 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.021153 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.021382 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.021469 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.021682 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.021780 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.021983 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.022082 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.022288 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.022373 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.022590 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.022687 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.022895 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.022981 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.023200 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.023285 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.023503 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.023598 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.023819 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.023905 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.024123 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.024208 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.024427 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.024514 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.024771 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.024860 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.025078 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.025164 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.025384 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.025472 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.025696 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.025831 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.026006 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.026101 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.026317 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.026410 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.026618 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.026704 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.026922 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.027015 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.027220 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.027315 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.027518 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.027604 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.027816 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.027911 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.028118 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.028204 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.028415 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.028500 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.028728 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.028833 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.029035 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.029121 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.029335 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.029421 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.029633 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.029728 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.029948 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.030034 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.030250 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.030342 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.030547 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.030642 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.030843 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.030930 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.031141 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.031226 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.031492 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.032130 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.032305 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.032392 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.032612 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.032715 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.032940 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.033031 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.033247 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.033340 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.033553 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.033639 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.033878 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.033966 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.034187 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.034273 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.034486 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.034572 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.034788 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.034881 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.035088 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.035173 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.035393 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.035535 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.035707 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.035794 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.036012 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.036106 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.036324 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.036419 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.036624 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.036746 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.036965 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.037051 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.037269 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.037370 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.037583 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.037669 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.037881 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.037974 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.038188 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.038284 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.038496 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.038591 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.038807 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.038892 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.039125 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.039226 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.039437 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.039531 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.039740 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.039825 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.040049 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.040144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.040351 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.040439 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.040655 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.040773 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.040986 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.041076 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.041288 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.041373 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.041599 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.041683 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.041901 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.041989 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.042210 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.042296 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.042510 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.042605 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.042824 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.042912 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.043125 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.043218 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.043436 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.043532 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.043732 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.043821 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.044039 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.044127 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.044357 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.044463 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.044661 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.044808 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.045054 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.045130 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.045365 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.045455 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.045747 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.045834 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.045985 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.046062 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.046288 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.046372 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.046590 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.046668 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.046892 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.046968 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.047221 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.047308 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.047521 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.047606 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.047831 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.047918 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.048133 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.048225 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.048433 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.048568 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.048751 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.048854 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.049059 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.049147 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.049360 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.049447 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.049656 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.049749 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.049954 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.050041 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.050258 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.050348 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.050559 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.050653 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.050860 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.050953 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.051173 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.051266 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.051482 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.051577 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.051777 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.051869 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.052073 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.052161 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.052382 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.052476 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.052681 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.052816 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053001 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053087 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053299 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053398 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053596 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053681 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053914 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.053998 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.054221 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.054307 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.054515 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.054671 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.054823 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.054910 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.055134 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.055228 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.055434 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.055519 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.055740 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.055853 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.056043 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.056151 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.056339 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.056432 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.056641 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.056750 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.056972 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.057061 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.057270 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.057354 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.057573 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.057658 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.057876 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.057963 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.058179 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.058265 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.058482 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.058590 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.058787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.058874 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.059084 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.059170 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.059386 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.059481 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.059688 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.059775 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.059984 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.060070 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.060285 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.060381 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.060591 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.060677 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.060927 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.061170 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.061256 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.061502 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.061599 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.061812 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.061897 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.062120 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.062223 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.062424 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.062515 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.062758 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.062858 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.063066 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.063152 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.063367 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.063456 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.063674 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.063760 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.063970 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.064054 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.064272 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.064545 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.064640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.064880 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.064971 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.065184 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.065285 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.065482 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.065569 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.065777 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.065863 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.066079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.066209 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.066379 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.066480 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.066676 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.066820 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.066973 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.067067 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.067269 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.067358 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.067565 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.067652 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.067870 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.067958 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.068165 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.068253 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.068469 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.068555 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.068788 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.068893 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.069105 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.069192 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.069405 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.069540 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.069702 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.069803 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.069996 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.070083 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.070291 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.070425 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.070592 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.070692 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.070894 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.070981 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.071204 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.071290 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.071506 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.071593 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.071816 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.071904 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.072116 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.072202 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.072416 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.072503 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.072749 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.072848 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.073051 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.073145 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.073348 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.073436 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.073643 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.073741 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.073942 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.074029 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.074238 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.074327 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.074531 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.074617 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.074834 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.074929 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.075136 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.075223 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.075435 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.075523 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.075733 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.075829 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.076025 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.076122 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.076325 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.076484 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.076617 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.076816 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.076954 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.077122 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.077237 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.077429 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.077535 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.077735 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.077830 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.078036 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.078129 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.078872 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.078991 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.079188 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.079333 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.079496 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.079583 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.079790 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.079887 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.080092 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.080186 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.080387 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.080521 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.080685 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.080857 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081001 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081089 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081303 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081389 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081601 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081694 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081902 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.081988 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.082209 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.082295 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.082511 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.082598 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.082811 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.082909 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.083111 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.083196 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.083413 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.083501 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.083713 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.083798 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.084014 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.084101 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.084315 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.084403 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.084616 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.084721 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.084948 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.085046 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.085244 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.085331 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.085541 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.085627 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.085836 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.085934 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.086133 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.086219 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.086431 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.086531 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.086728 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.086824 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087023 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087110 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087318 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087406 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087614 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087908 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.087995 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.088203 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.088288 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.088514 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.088613 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.088836 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.088938 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.089134 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.089232 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.089426 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.089516 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.089721 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.089808 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.090018 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.090103 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.090323 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.090410 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.090623 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.090709 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.090923 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.091008 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.091222 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.091313 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.091522 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.091652 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.091823 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.091917 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.092140 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.092228 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.092445 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.092530 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.092762 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.092938 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.093065 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.093153 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.093363 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.093491 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.093666 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.093760 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.094508 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.094610 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.094818 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.094905 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.095133 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.095234 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.095442 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.095529 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.095747 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.095833 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.096061 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.096156 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.096375 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.096468 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.096692 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.096792 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.097014 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.097104 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.097317 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.097393 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.097623 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.097708 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.097924 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.098002 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.098224 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.098299 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.098523 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.098597 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.098830 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.098907 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.099212 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.099310 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.099441 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.099517 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.099744 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.099822 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.100049 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.100124 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.100347 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.100423 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.100687 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.100809 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.100974 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.101051 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.101275 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.101360 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.101576 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.101651 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.101885 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.101960 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.102181 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.102266 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.102483 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.102558 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.102779 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.102854 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.103085 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.103181 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.103382 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.103467 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.103688 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.103801 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.103993 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.104088 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.104301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.104388 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.104608 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.104714 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.104940 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.105029 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.105263 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.105350 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.105572 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.105665 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.105875 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.105963 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.106171 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.106256 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.106472 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.106580 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.106785 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.106885 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.107084 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.107181 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.107467 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.107554 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.107765 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.107854 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.108062 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.108151 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.108365 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.108461 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.108681 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.108829 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.109000 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.109097 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.109301 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.109432 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.109626 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.109728 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.109926 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.110010 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.110225 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.110342 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.110536 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.110655 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.110943 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.111051 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.111243 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.111339 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.111541 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.111661 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.111840 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.112838 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.112973 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113092 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113202 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113317 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113439 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113548 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113664 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.113891 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114000 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114108 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114215 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114332 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114489 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114605 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114736 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114856 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.114967 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115075 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115190 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115308 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115418 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115525 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115631 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115813 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.115933 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.116599 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.116689 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.116849 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.116935 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.117023 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.117094 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.117179 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.117266 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.117432 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.117546 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.117748 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.118094 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.118301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.118387 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.118614 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.118701 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.118934 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.119031 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.119230 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.119317 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.119530 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.119630 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.119828 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.119916 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.120124 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.120219 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.120425 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.120526 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.120765 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.120874 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.121067 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.121157 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.121372 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.121474 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.121682 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.121792 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.121983 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.122102 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.122280 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.122420 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.122577 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.122740 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.122892 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.123042 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.123192 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.123344 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.123507 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.123650 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.123812 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.123951 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.124126 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.124253 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.124437 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.124571 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.124769 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.124901 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.125062 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.125234 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.125391 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.125545 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.125690 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.125849 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.125987 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.126148 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.126283 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.126448 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.126618 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.126764 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.126918 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.127064 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.127232 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.127381 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.127558 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.127684 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.127880 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.127981 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.128183 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.128295 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.128507 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.128603 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.128848 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.128940 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.129165 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.129251 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.129469 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.129588 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.129783 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.129882 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.130095 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.130180 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.130390 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.130490 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.130686 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.130773 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.130987 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.131130 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.131286 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.131376 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.131582 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.131669 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.131884 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.131973 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.132174 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.132274 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.132466 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.132634 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.132784 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.132979 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.133080 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.133288 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.133376 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.133601 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.133688 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.133921 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.134020 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.134224 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.134320 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.134525 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.134613 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.134820 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.134908 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.135115 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.135202 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.135414 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.135500 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.135715 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.135805 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.136018 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.136104 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.136317 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.136412 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.136616 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.136725 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.136937 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.137037 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.137256 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.137343 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.137556 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.137704 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.137864 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.137950 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.138165 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.138260 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.138465 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.138562 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.138764 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.138851 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.139060 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.139156 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.139365 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.139459 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.139675 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.139766 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.139986 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.140087 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.140287 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.140511 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.140600 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.141391 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.141504 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.141704 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.141794 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.142010 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.142098 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.142322 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.142420 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.142630 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.142718 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.142937 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.143033 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.143242 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.143342 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.143545 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.143633 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.143846 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.143940 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.144149 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.144252 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.144460 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.144547 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.144804 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.145022 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.145124 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.145357 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.145446 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.145671 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.145758 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.145980 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.146075 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.146289 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.146377 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.146600 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.146686 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.146909 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.147007 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.147216 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.147305 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.147522 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.147611 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.147829 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.147918 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.148137 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.148223 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.148450 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.148537 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.148784 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.148887 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.149096 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.149191 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.149408 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.149495 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.149718 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.149807 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.150022 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.150109 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.150326 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.150422 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.150636 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.150723 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.150945 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.151039 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.151261 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.151363 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.151577 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.151663 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.151890 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.151976 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.152205 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.152341 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.152516 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.152609 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.152847 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.152936 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.153158 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.153363 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.153457 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.153674 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.153761 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.154001 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.154097 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.154321 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.154409 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.154638 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.154725 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.154947 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.155039 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.155258 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.155351 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.155578 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.155666 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.155879 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.155968 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.156178 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.156271 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.156541 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.156642 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.156872 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.156971 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.157172 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.157385 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.157479 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.157697 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.157795 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.158025 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.158116 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.158346 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.158443 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.158661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.158748 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.158967 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.159184 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.159270 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.159522 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.159618 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.159831 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.159931 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.160149 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.160243 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.160455 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.160554 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.160789 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.160881 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.161091 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.161176 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.161405 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.161574 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.161705 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.161793 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.162003 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.162088 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.162316 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.162451 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.162618 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.162712 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.162917 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.163141 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.163237 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.163482 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.163570 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.163791 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.163876 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.164090 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.164186 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.164394 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.164481 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.164690 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.164794 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.165028 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.165128 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.165329 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.165415 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.165627 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.165713 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.165926 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.166014 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.166232 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.166319 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.166532 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.166618 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.166830 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.166918 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.167129 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.167215 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.167427 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.167513 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.167729 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.167818 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.168029 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.168128 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.168328 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.168426 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.168626 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.168750 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.168953 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.169052 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.169266 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.169364 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.169570 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.169657 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.169869 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.169956 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.170178 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.170273 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.170478 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.170563 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.170774 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.170867 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.171073 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.171168 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.171374 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.171461 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.171678 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.171762 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.171978 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.172578 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.172819 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.172909 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.173124 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.173209 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.173433 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.173522 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.173782 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.173963 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.174087 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.174292 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.174388 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.174605 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.174692 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.174931 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.175017 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.175253 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.175349 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.175554 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.175642 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.175854 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.175940 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.176154 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.176249 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.176464 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.176549 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.176789 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.176877 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.177099 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.177189 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.177405 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.177512 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.177708 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.178026 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.178177 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.178315 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.178405 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.178617 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.178703 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.178918 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.179014 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.179219 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.179325 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.179522 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.179609 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.179825 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.179914 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.180126 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.180213 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.180427 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.180514 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.180746 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.180841 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.181058 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.181144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.181390 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.181477 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.181704 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.181834 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.182011 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.182097 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.182316 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.182455 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.182623 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.182757 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.182924 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.183011 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.183225 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.183324 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.183527 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.183615 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.183828 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.183921 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.184131 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.184226 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.184440 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.184528 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.184766 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.184856 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.185068 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.185171 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.185406 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.185728 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.185817 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.186041 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.186137 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.186349 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.186435 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.186663 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.186748 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.186968 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.187065 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.187273 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.187366 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.187579 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.187686 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.187937 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.188032 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.188251 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.188336 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.188552 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.188638 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.188900 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.188991 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.189204 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.189290 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.189540 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.189855 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.190001 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.190153 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.190242 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.190449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.190537 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.190747 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.190907 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.191050 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.191138 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.191361 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.191448 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.191661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.191790 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.191956 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.192047 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.192253 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.192342 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.192558 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.192647 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.192880 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.192981 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.193175 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.193263 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.193473 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.193558 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.193772 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.193865 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.194070 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.194165 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.194366 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.194454 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.194663 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.194757 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.194960 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.195055 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.195254 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.195341 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.195547 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.195635 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.195842 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.195937 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.196142 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.196228 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.196436 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.196525 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.196751 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.196856 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.197051 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.197139 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.197348 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.197442 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.197643 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.197732 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.197946 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.198032 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.198245 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.198382 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.198546 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.198637 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.198847 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.198934 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.199147 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.199233 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.199447 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.199548 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.199746 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.199841 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.200051 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.200146 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.200350 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.200438 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.200651 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.200759 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.200972 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.201076 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.201274 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.201362 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.201573 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.201662 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.201879 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.202015 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.202189 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.202278 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.202496 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.202622 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.202800 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.202898 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.203103 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.203189 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.203980 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.204077 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.204246 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.204344 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.204546 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.204631 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.204897 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.205001 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.205215 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.205304 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.205527 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.205620 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.205858 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.205944 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.206168 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.206264 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.206484 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.206578 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.206798 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.206883 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.207104 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.207193 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.207405 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.207491 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.207710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.207804 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.208011 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.208105 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.208321 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.208458 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.208626 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.208745 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.208958 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.209045 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.209269 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.209356 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.209577 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.209672 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.209890 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.209975 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.210201 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.210300 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.210503 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.210630 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.210826 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.210913 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.211136 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.211223 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.211441 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.211537 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.211750 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.211837 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.212058 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.212144 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.212370 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.212466 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.212677 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.212796 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.213006 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.213107 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.213307 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.213411 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.213547 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 02:20:56.213635 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.213926 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.214249 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.214556 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.214879 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.215200 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.215542 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.215853 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.216197 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.216523 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.216861 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.217173 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.217478 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:20:56.217718 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 02:20:56.365635 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO dbcounter [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Registered counter for database nova_api Oct 12 02:20:56.371024 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO dbcounter [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Registered counter for database nova_api Oct 12 02:20:56.386810 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_db.sqlalchemy.engines [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.391728 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_db.sqlalchemy.engines [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.394903 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writer thread running {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.399050 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writer thread running {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.450029 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.context [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dbde3dce-57ec-471e-b3b0-530d5203c2f2(cell1) {{(pid=103653) load_cells /opt/stack/nova/nova/context.py:464}} Oct 12 02:20:56.454548 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_concurrency.lockutils [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:20:56.454945 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_concurrency.lockutils [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:20:56.455975 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.context [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dbde3dce-57ec-471e-b3b0-530d5203c2f2(cell1) {{(pid=103654) load_cells /opt/stack/nova/nova/context.py:464}} Oct 12 02:20:56.456474 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_concurrency.lockutils [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:20:56.457230 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_concurrency.lockutils [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:20:56.457673 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_concurrency.lockutils [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:20:56.459614 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_concurrency.lockutils [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:20:56.459819 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:20:56.460297 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:20:56.461582 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO dbcounter [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Registered counter for database nova_cell0 Oct 12 02:20:56.461970 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:20:56.462695 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:20:56.463153 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:20:56.465170 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:20:56.465627 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO dbcounter [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Registered counter for database nova_cell1 Oct 12 02:20:56.467148 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO dbcounter [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Registered counter for database nova_cell0 Oct 12 02:20:56.471224 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO dbcounter [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Registered counter for database nova_cell1 Oct 12 02:20:56.472923 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_db.sqlalchemy.engines [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.475115 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_db.sqlalchemy.engines [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.478788 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_db.sqlalchemy.engines [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.480769 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writer thread running {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.480948 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_db.sqlalchemy.engines [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.482229 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writer thread running {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.486516 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writer thread running {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.489313 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writer thread running {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.623929 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO dbcounter [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Registered counter for database nova_cell0 Oct 12 02:20:56.632581 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO dbcounter [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Registered counter for database nova_cell0 Oct 12 02:20:56.632760 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG oslo_db.sqlalchemy.engines [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.637289 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writer thread running {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.639468 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_db.sqlalchemy.engines [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:20:56.644637 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writer thread running {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:20:56.969910 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: WARNING keystonemiddleware.auth_token [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:20:56.976332 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: WARNING keystonemiddleware.auth_token [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:20:56.977154 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: WARNING keystonemiddleware.auth_token [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:20:56.983679 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: WARNING keystonemiddleware.auth_token [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:20:57.320066 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: WARNING keystonemiddleware.auth_token [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:20:57.323056 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: WARNING keystonemiddleware.auth_token [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:20:57.326042 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: WARNING keystonemiddleware.auth_token [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:20:57.329073 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: WARNING keystonemiddleware.auth_token [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:20:57.332171 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a6869cb9c0 pid: 103653 (default app) Oct 12 02:20:57.334313 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:20:57.334446 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a6869cb9c0 pid: 103654 (default app) Oct 12 02:20:57.335499 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-4cb8a798-4727-41ca-8b9b-84e1ad9f61f5 None None] 10.222.0.23 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001751 Oct 12 02:20:57.335922 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 788 bytes} [Wed Oct 12 02:20:57 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:17.605620 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:17.606844 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002545 Oct 12 02:22:17.607254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:17 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:17.618486 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_api:SELECT=3 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:17.619605 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_cell1:SELECT=3 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:17.620460 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_cell0:SELECT=3 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:17.622513 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:18.474210 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-f76cc0ce-0578-4607-8cbc-485b8c20ff5d admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:18.506932 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-f76cc0ce-0578-4607-8cbc-485b8c20ff5d admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.893758 Oct 12 02:22:18.507802 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 2/3] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 12 02:22:17 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 895 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:21.139517 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:21.140608 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-1ed1690a-d90d-4574-8d5b-0b5cb6a59a3e None None] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001566 Oct 12 02:22:21.146335 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 2/4] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:21 2022] GET /compute/v2.1 => generated 388 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:21.590531 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-48a51382-31f1-4e26-8094-9e0886675d3e admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "cirros256", "vcpus": 1, "disk": 1, "ram": 256, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "c1"}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:21.636580 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-48a51382-31f1-4e26-8094-9e0886675d3e admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.489067 Oct 12 02:22:21.637279 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 3/5] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:21 2022] POST /compute/v2.1/flavors => generated 407 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 12 02:22:21.651408 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_api:SELECT=3 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:21.652844 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_cell0:SELECT=3 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:21.654819 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_cell1:SELECT=3 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:21.655617 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:21.862487 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-10ccb2c0-3bca-4eb1-9fba-47055ba08c03 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:21.929107 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-10ccb2c0-3bca-4eb1-9fba-47055ba08c03 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.283397 Oct 12 02:22:21.929692 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 3/6] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 12 02:22:21 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:24.614120 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-48a51382-31f1-4e26-8094-9e0886675d3e admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:24.615392 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-48a51382-31f1-4e26-8094-9e0886675d3e admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001707 Oct 12 02:22:24.615790 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 4/7] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:24 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:25.262439 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2c777001-3ee4-415e-b0a0-2a208acdf3bc admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "id": "d1", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 1, "disk": 5, "name": "ds512M", "swap": 0, "ram": 512}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:25.303183 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-2c777001-3ee4-415e-b0a0-2a208acdf3bc admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.681164 Oct 12 02:22:25.303931 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 4/8] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:24 2022] POST /compute/v2.1/flavors => generated 404 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 12 02:22:25.315851 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-ff8940b5-a3f7-4d7e-a43a-92e82f806ebd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:25.361511 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-ff8940b5-a3f7-4d7e-a43a-92e82f806ebd admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.049242 Oct 12 02:22:25.362100 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 5/9] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 12 02:22:25 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:27.948378 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2c777001-3ee4-415e-b0a0-2a208acdf3bc admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:27.949445 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-2c777001-3ee4-415e-b0a0-2a208acdf3bc admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002101 Oct 12 02:22:27.949846 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:27 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:28.364359 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-869d6c6d-e003-4d27-8642-5f93b587c8bd admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 10, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 1024, "id": "d2", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "name": "ds1G"}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:28.385463 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-869d6c6d-e003-4d27-8642-5f93b587c8bd admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.428556 Oct 12 02:22:28.386323 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 6/11] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:27 2022] POST /compute/v2.1/flavors => generated 404 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:22:28.397892 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-683601b3-dd55-409a-9ce4-3f7a90e3b690 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:28.425714 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-683601b3-dd55-409a-9ce4-3f7a90e3b690 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030653 Oct 12 02:22:28.426405 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 6/12] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 12 02:22:28 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:31.013100 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-869d6c6d-e003-4d27-8642-5f93b587c8bd admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:31.014718 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-869d6c6d-e003-4d27-8642-5f93b587c8bd admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002075 Oct 12 02:22:31.025471 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 7/13] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:31 2022] GET /compute/v2.1 => generated 388 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:31.470143 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-b898a167-90c0-48ee-aa2f-383d908d3cbb admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "os-flavor-access:is_public": true, "id": "d3", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "name": "ds2G", "disk": 10, "swap": 0}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:31.489840 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-b898a167-90c0-48ee-aa2f-383d908d3cbb admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.468116 Oct 12 02:22:31.493053 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 7/14] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:31 2022] POST /compute/v2.1/flavors => generated 404 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:22:31.502908 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-7151be7e-dc9d-4576-9682-e3bb6b920452 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:31.529790 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-7151be7e-dc9d-4576-9682-e3bb6b920452 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030493 Oct 12 02:22:31.530563 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 8/15] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 12 02:22:31 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:34.363550 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-b898a167-90c0-48ee-aa2f-383d908d3cbb admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:34.364957 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-b898a167-90c0-48ee-aa2f-383d908d3cbb admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001532 Oct 12 02:22:34.365289 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 8/16] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:34.826100 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-0032d601-22ac-431c-89c8-f7ab89efaa11 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d4", "vcpus": 4, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "name": "ds4G", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 4096, "disk": 20}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:34.845771 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-0032d601-22ac-431c-89c8-f7ab89efaa11 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.470040 Oct 12 02:22:34.846319 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 9/17] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:34 2022] POST /compute/v2.1/flavors => generated 404 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:22:34.864296 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-6e101f94-9234-4027-8d79-b9d39a66f3d8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:34.892288 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-6e101f94-9234-4027-8d79-b9d39a66f3d8 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031649 Oct 12 02:22:34.892862 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 9/18] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 12 02:22:34 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:37.924509 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-0032d601-22ac-431c-89c8-f7ab89efaa11 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:37.925575 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-0032d601-22ac-431c-89c8-f7ab89efaa11 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001508 Oct 12 02:22:37.925990 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 10/19] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:38.344921 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-d1e755a7-a7d9-4259-98f2-67b7cd196047 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:38.379195 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-d1e755a7-a7d9-4259-98f2-67b7cd196047 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.446417 Oct 12 02:22:38.379652 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 10/20] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 12 02:22:37 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:22:40.969355 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-0032d601-22ac-431c-89c8-f7ab89efaa11 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:40.970369 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-0032d601-22ac-431c-89c8-f7ab89efaa11 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001490 Oct 12 02:22:40.970773 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 11/21] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:41.380464 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-4acfaf58-b603-44a9-8c6f-be3a5d9a7144 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "rxtx_factor": 1.0, "name": "m1.tiny", "swap": 0, "os-flavor-access:is_public": true, "id": "1", "vcpus": 1, "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:41.401999 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-4acfaf58-b603-44a9-8c6f-be3a5d9a7144 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.422911 Oct 12 02:22:41.402545 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 11/22] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:40 2022] POST /compute/v2.1/flavors => generated 402 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:22:41.414428 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-0c3015c1-e3f6-4ccb-a05d-a2347a93ba64 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:41.441916 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-0c3015c1-e3f6-4ccb-a05d-a2347a93ba64 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031073 Oct 12 02:22:41.442503 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 12/23] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 12 02:22:41 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:44.026353 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-4acfaf58-b603-44a9-8c6f-be3a5d9a7144 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:44.027370 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-4acfaf58-b603-44a9-8c6f-be3a5d9a7144 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Oct 12 02:22:44.027777 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:44.426680 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-427eed6c-a1e5-474c-99df-5db3f7a8c416 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "ram": 2048, "disk": 20, "os-flavor-access:is_public": true, "vcpus": 1, "name": "m1.small", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "2", "rxtx_factor": 1.0}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:44.451091 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-427eed6c-a1e5-474c-99df-5db3f7a8c416 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.416042 Oct 12 02:22:44.451884 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 13/25] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:44 2022] POST /compute/v2.1/flavors => generated 405 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:22:44.468881 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-c909d58d-c2c4-4357-8e27-992172e043b7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:44.496735 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-c909d58d-c2c4-4357-8e27-992172e043b7 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033015 Oct 12 02:22:44.497661 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 13/26] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 12 02:22:44 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:47.152242 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-427eed6c-a1e5-474c-99df-5db3f7a8c416 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:47.153298 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-427eed6c-a1e5-474c-99df-5db3f7a8c416 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001495 Oct 12 02:22:47.153712 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 14/27] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:47.594483 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-40f4ec24-eeeb-470a-b10c-0e590ce16806 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 40, "id": "3", "swap": 0, "name": "m1.medium", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 2, "ram": 4096}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:47.617730 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-40f4ec24-eeeb-470a-b10c-0e590ce16806 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.457921 Oct 12 02:22:47.618797 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 14/28] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:47 2022] POST /compute/v2.1/flavors => generated 406 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:22:47.631065 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-2ce7ba14-8236-4ab5-b323-8fa4a87ed0cf admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:47.658543 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-2ce7ba14-8236-4ab5-b323-8fa4a87ed0cf admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031445 Oct 12 02:22:47.659739 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 15/29] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 12 02:22:47 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:50.249621 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-40f4ec24-eeeb-470a-b10c-0e590ce16806 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:50.251425 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-40f4ec24-eeeb-470a-b10c-0e590ce16806 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002253 Oct 12 02:22:50.252422 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 15/30] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:50 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:50.684333 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-cedc08ca-e545-4e1d-bfef-64827faf9449 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "4", "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "os-flavor-access:is_public": true, "disk": 80, "swap": 0, "ram": 8192, "rxtx_factor": 1.0, "vcpus": 4}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:50.691695 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:50.707552 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-cedc08ca-e545-4e1d-bfef-64827faf9449 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.449276 Oct 12 02:22:50.711805 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 16/31] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:50 2022] POST /compute/v2.1/flavors => generated 405 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 12 02:22:50.720809 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-efa61494-8301-406d-8065-41636814b816 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:50.844172 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-efa61494-8301-406d-8065-41636814b816 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.127183 Oct 12 02:22:50.845015 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 16/32] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 12 02:22:50 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:22:53.432185 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-cedc08ca-e545-4e1d-bfef-64827faf9449 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:22:53.433128 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-cedc08ca-e545-4e1d-bfef-64827faf9449 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 Oct 12 02:22:53.433529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 17/33] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:22:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:22:53.874245 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-24c5dbba-0d8a-4b05-a609-05ce13073bc6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "id": "5", "name": "m1.xlarge", "rxtx_factor": 1.0, "vcpus": 8, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 160, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:53.881068 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:22:53.897943 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-24c5dbba-0d8a-4b05-a609-05ce13073bc6 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.458445 Oct 12 02:22:53.898541 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 17/34] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 12 02:22:53 2022] POST /compute/v2.1/flavors => generated 408 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:22:53.910737 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-260558d6-d5c4-46dc-83e9-4b9f8138dbf6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:22:53.937485 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-260558d6-d5c4-46dc-83e9-4b9f8138dbf6 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030475 Oct 12 02:22:53.938333 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 18/35] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 12 02:22:53 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:23:11.390985 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-24c5dbba-0d8a-4b05-a609-05ce13073bc6 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:11.391982 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-24c5dbba-0d8a-4b05-a609-05ce13073bc6 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001444 Oct 12 02:23:11.392379 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 18/36] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:23:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:11.403219 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:23:11.775121 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-54cc67ae-8682-40af-99c5-04eab2ab8846 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:11.794658 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-54cc67ae-8682-40af-99c5-04eab2ab8846 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.396283 Oct 12 02:23:11.795573 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 19/37] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 12 02:23:11 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:23:13.087295 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-24c5dbba-0d8a-4b05-a609-05ce13073bc6 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:13.088381 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-24c5dbba-0d8a-4b05-a609-05ce13073bc6 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001496 Oct 12 02:23:13.088800 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 19/38] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:13.605537 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-cf99ccb8-49c2-4455-bb77-00a50c553178 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:13.610741 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.wsgi [None req-cf99ccb8-49c2-4455-bb77-00a50c553178 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-cf99ccb8-49c2-4455-bb77-00a50c553178 admin admin] Request method failure captured: Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Accept: application/json Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Accept-Encoding: gzip, deflate Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Connection: keep-alive Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Content-Length: 0 Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Host: 10.222.0.23 Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Openstack-Api-Version: compute 2.92 Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Openstack-System-Scope: None Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: User-Agent: python-novaclient Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Auth-Token: gAAAAABjRiUQSoKRrV1qvFcZCZxSxi6kKsNL50mk2SlKHORlL5ka2UaM9uCy5oN6laF8S60gk8f74BZ31-OwrQDOn2rmXIFMOM224fw_nIBfIz3YTjxKAKD__cVUPFnq7BISHavifr-kNcWp5pdfyCfRfYQ0rtVUkEp80OuO4GeZQFrQ48ZhzXA Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Domain-Id: None Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Domain-Name: None Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Identity-Status: Confirmed Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Is-Admin-Project: True Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Openstack-Nova-Api-Version: 2.92 Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Domain-Id: default Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Domain-Name: Default Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Name: admin Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Role: member,admin,reader Oct 12 02:23:13.614227 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Roles: member,admin,reader Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_1b152cd4fded41fa8ba95182a5dc98e4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant: admin Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant-Name: admin Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User: admin Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Domain-Id: default Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Domain-Name: Default Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Id: 1bca8e4c760d498693a8dcfc0e178249 Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Name: admin Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: method: > Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: exception: Flavor 42 could not be found. Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: action_args: {'id': '42'} Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:23:13.615806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:13.617256 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.618449 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-cf99ccb8-49c2-4455-bb77-00a50c553178 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:23:13.619417 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-cf99ccb8-49c2-4455-bb77-00a50c553178 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.389099 Oct 12 02:23:13.619417 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 20/39] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 12 02:23:13 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 390 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 12 02:23:13.626955 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-e7f1f918-13b8-4d55-89a7-c0f5e82dccca admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:13.648917 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-e7f1f918-13b8-4d55-89a7-c0f5e82dccca admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026353 Oct 12 02:23:13.649440 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 20/40] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 12 02:23:13 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:23:13.659939 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-93f7a7f9-2504-4683-99cc-d91d02fc2422 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:13.682079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-93f7a7f9-2504-4683-99cc-d91d02fc2422 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026612 Oct 12 02:23:13.682602 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 21/41] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 12 02:23:13 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:23:13.691683 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-e56da977-2e91-40ab-b17e-a4ef5090d549 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:13.697491 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.wsgi [None req-e56da977-2e91-40ab-b17e-a4ef5090d549 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-e56da977-2e91-40ab-b17e-a4ef5090d549 admin admin] Request method failure captured: Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Accept: application/json Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Accept-Encoding: gzip, deflate Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Connection: keep-alive Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Content-Length: 0 Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Host: 10.222.0.23 Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Openstack-Api-Version: compute 2.92 Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Openstack-System-Scope: None Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: User-Agent: python-novaclient Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Auth-Token: gAAAAABjRiUQSoKRrV1qvFcZCZxSxi6kKsNL50mk2SlKHORlL5ka2UaM9uCy5oN6laF8S60gk8f74BZ31-OwrQDOn2rmXIFMOM224fw_nIBfIz3YTjxKAKD__cVUPFnq7BISHavifr-kNcWp5pdfyCfRfYQ0rtVUkEp80OuO4GeZQFrQ48ZhzXA Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Domain-Id: None Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Domain-Name: None Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Identity-Status: Confirmed Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Is-Admin-Project: True Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Openstack-Nova-Api-Version: 2.92 Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Domain-Id: default Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Domain-Name: Default Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Name: admin Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Role: member,admin,reader Oct 12 02:23:13.700787 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Roles: member,admin,reader Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_1b152cd4fded41fa8ba95182a5dc98e4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant: admin Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant-Name: admin Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User: admin Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Domain-Id: default Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Domain-Name: Default Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Id: 1bca8e4c760d498693a8dcfc0e178249 Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Name: admin Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: method: > Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: exception: Flavor 42 could not be found. Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: action_args: {'id': '42'} Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:23:13.702292 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:13.703254 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:13.704509 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-e56da977-2e91-40ab-b17e-a4ef5090d549 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:23:13.705758 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-e56da977-2e91-40ab-b17e-a4ef5090d549 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014001 Oct 12 02:23:13.705758 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 21/42] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 12 02:23:13 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 12 02:23:14.912414 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-93f7a7f9-2504-4683-99cc-d91d02fc2422 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:14.913466 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-93f7a7f9-2504-4683-99cc-d91d02fc2422 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001503 Oct 12 02:23:14.913860 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 22/43] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:15.431323 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2f2fd0e3-dfd6-4b86-ab11-10a04ec78840 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:15.436331 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.wsgi [None req-2f2fd0e3-dfd6-4b86-ab11-10a04ec78840 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2f2fd0e3-dfd6-4b86-ab11-10a04ec78840 admin admin] Request method failure captured: Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Accept: application/json Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Accept-Encoding: gzip, deflate Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Connection: keep-alive Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Content-Length: 0 Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Host: 10.222.0.23 Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Openstack-Api-Version: compute 2.92 Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Openstack-System-Scope: None Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: User-Agent: python-novaclient Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Auth-Token: gAAAAABjRiUSilUqkrR0LyyvmeNPeszqiZgB8J8plJA67EtrUcRJLtXcdFOeT17jsuz1YH3Xbqc1ASJWfygieZAI2VvidHvWUHFVa-KzlORWm8Qc4yTvc0r132gyHRH2i2SGoCsk3N9WfcrEw73gv3fL1lxF7yCohT2zbVjuEzGQ-erFdcz6yEg Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Domain-Id: None Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Domain-Name: None Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Identity-Status: Confirmed Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Is-Admin-Project: True Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Openstack-Nova-Api-Version: 2.92 Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Domain-Id: default Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Domain-Name: Default Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Name: admin Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Role: member,admin,reader Oct 12 02:23:15.437848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Roles: member,admin,reader Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_1b152cd4fded41fa8ba95182a5dc98e4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant: admin Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant-Name: admin Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User: admin Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Domain-Id: default Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Domain-Name: Default Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Id: 1bca8e4c760d498693a8dcfc0e178249 Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Name: admin Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: method: > Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: exception: Flavor 84 could not be found. Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: action_args: {'id': '84'} Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:23:15.439301 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:15.440264 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.441533 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2f2fd0e3-dfd6-4b86-ab11-10a04ec78840 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:23:15.442749 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-2f2fd0e3-dfd6-4b86-ab11-10a04ec78840 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.387042 Oct 12 02:23:15.442749 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 12 02:23:15 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 388 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 12 02:23:15.448969 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-394cb7b7-ba49-4d86-b35a-eaf22a6515c3 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:15.463543 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-394cb7b7-ba49-4d86-b35a-eaf22a6515c3 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018094 Oct 12 02:23:15.464056 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 23/45] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 12 02:23:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:23:15.473234 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-37c3d024-5c69-4dfd-afd0-0ff1566e0af1 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:15.488779 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-37c3d024-5c69-4dfd-afd0-0ff1566e0af1 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019045 Oct 12 02:23:15.489275 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 12 02:23:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:23:15.498781 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-52a7b78c-bdff-4279-9361-97059c8b5611 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:15.503691 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.wsgi [None req-52a7b78c-bdff-4279-9361-97059c8b5611 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-52a7b78c-bdff-4279-9361-97059c8b5611 admin admin] Request method failure captured: Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Accept: application/json Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Accept-Encoding: gzip, deflate Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Connection: keep-alive Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Content-Length: 0 Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Host: 10.222.0.23 Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Openstack-Api-Version: compute 2.92 Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Openstack-System-Scope: None Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: User-Agent: python-novaclient Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Auth-Token: gAAAAABjRiUSilUqkrR0LyyvmeNPeszqiZgB8J8plJA67EtrUcRJLtXcdFOeT17jsuz1YH3Xbqc1ASJWfygieZAI2VvidHvWUHFVa-KzlORWm8Qc4yTvc0r132gyHRH2i2SGoCsk3N9WfcrEw73gv3fL1lxF7yCohT2zbVjuEzGQ-erFdcz6yEg Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Domain-Id: None Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Domain-Name: None Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Identity-Status: Confirmed Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Is-Admin-Project: True Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Openstack-Nova-Api-Version: 2.92 Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Domain-Id: default Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Domain-Name: Default Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Name: admin Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Role: member,admin,reader Oct 12 02:23:15.505214 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Roles: member,admin,reader Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_1b152cd4fded41fa8ba95182a5dc98e4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant: admin Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant-Name: admin Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User: admin Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Domain-Id: default Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Domain-Name: Default Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Id: 1bca8e4c760d498693a8dcfc0e178249 Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Name: admin Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: method: > Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: exception: Flavor 84 could not be found. Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: action_args: {'id': '84'} Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:23:15.506661 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:15.507776 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:15.509079 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-52a7b78c-bdff-4279-9361-97059c8b5611 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:23:15.510254 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-52a7b78c-bdff-4279-9361-97059c8b5611 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011256 Oct 12 02:23:15.510254 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 24/47] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 12 02:23:15 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 12 02:23:16.711758 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-37c3d024-5c69-4dfd-afd0-0ff1566e0af1 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:16.712807 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-37c3d024-5c69-4dfd-afd0-0ff1566e0af1 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001425 Oct 12 02:23:16.713208 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 24/48] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:16 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:17.233833 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-195bdb08-aa95-41a5-863a-389fba104e61 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:17.240008 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.wsgi [None req-195bdb08-aa95-41a5-863a-389fba104e61 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-195bdb08-aa95-41a5-863a-389fba104e61 admin admin] Request method failure captured: Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Accept: application/json Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Accept-Encoding: gzip, deflate Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Connection: keep-alive Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Content-Length: 0 Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Host: 10.222.0.23 Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Openstack-Api-Version: compute 2.92 Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Openstack-System-Scope: None Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: User-Agent: python-novaclient Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Auth-Token: gAAAAABjRiUUBVWe0YviqwMPQ550CEMkAIsTEEdfSYW_NP3HSNX7w7IhHcHgyTMDx6G8693n6qItzK4leuQPrpVfE2mJyXD1XXcI7rasnuPF8tWSSbM0eWMvCKc9lU-ZkAhMKzsKL6HP2fdtOEBw3x9HARgRzTCbWIbFrMzUCmyVwzktwBKm-Mg Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Domain-Id: None Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Domain-Name: None Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Identity-Status: Confirmed Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Is-Admin-Project: True Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Openstack-Nova-Api-Version: 2.92 Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Domain-Id: default Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Domain-Name: Default Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Project-Name: admin Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Role: member,admin,reader Oct 12 02:23:17.242085 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Roles: member,admin,reader Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_1b152cd4fded41fa8ba95182a5dc98e4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant: admin Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-Tenant-Name: admin Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User: admin Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Domain-Id: default Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Domain-Name: Default Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Id: 1bca8e4c760d498693a8dcfc0e178249 Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: X-User-Name: admin Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: method: > Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: exception: Flavor 451 could not be found. Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: action_args: {'id': '451'} Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:23:17.243581 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:17.244529 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.245640 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-195bdb08-aa95-41a5-863a-389fba104e61 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:23:17.246692 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-195bdb08-aa95-41a5-863a-389fba104e61 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.393562 Oct 12 02:23:17.246692 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 25/49] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 12 02:23:16 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 395 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 12 02:23:17.253272 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-8fd1f7d6-cd0f-493d-9531-bb64a952ca7d admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:17.268555 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-8fd1f7d6-cd0f-493d-9531-bb64a952ca7d admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018837 Oct 12 02:23:17.269138 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 25/50] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 12 02:23:17 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:23:17.278783 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:17.294336 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019013 Oct 12 02:23:17.294850 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 26/51] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 12 02:23:17 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:23:17.303848 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2f278288-6edb-4ed2-8e83-6c172f7a5cd7 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:17.308642 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.wsgi [None req-2f278288-6edb-4ed2-8e83-6c172f7a5cd7 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2f278288-6edb-4ed2-8e83-6c172f7a5cd7 admin admin] Request method failure captured: Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Accept: application/json Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Accept-Encoding: gzip, deflate Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Connection: keep-alive Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Content-Length: 0 Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Host: 10.222.0.23 Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Openstack-Api-Version: compute 2.92 Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Openstack-System-Scope: None Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: User-Agent: python-novaclient Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Auth-Token: gAAAAABjRiUUBVWe0YviqwMPQ550CEMkAIsTEEdfSYW_NP3HSNX7w7IhHcHgyTMDx6G8693n6qItzK4leuQPrpVfE2mJyXD1XXcI7rasnuPF8tWSSbM0eWMvCKc9lU-ZkAhMKzsKL6HP2fdtOEBw3x9HARgRzTCbWIbFrMzUCmyVwzktwBKm-Mg Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Domain-Id: None Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Domain-Name: None Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Identity-Status: Confirmed Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Is-Admin-Project: True Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Openstack-Nova-Api-Version: 2.92 Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Domain-Id: default Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Domain-Name: Default Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Project-Name: admin Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Role: member,admin,reader Oct 12 02:23:17.310144 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Roles: member,admin,reader Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/1b152cd4fded41fa8ba95182a5dc98e4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_1b152cd4fded41fa8ba95182a5dc98e4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant: admin Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant-Id: 1b152cd4fded41fa8ba95182a5dc98e4 Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-Tenant-Name: admin Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User: admin Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Domain-Id: default Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Domain-Name: Default Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Id: 1bca8e4c760d498693a8dcfc0e178249 Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: X-User-Name: admin Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: method: > Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: exception: Flavor 451 could not be found. Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: action_args: {'id': '451'} Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:23:17.311710 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:17.312676 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: ERROR nova.api.openstack.wsgi Oct 12 02:23:17.314059 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-2f278288-6edb-4ed2-8e83-6c172f7a5cd7 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:23:17.315256 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-2f278288-6edb-4ed2-8e83-6c172f7a5cd7 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011081 Oct 12 02:23:17.315256 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 26/52] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 12 02:23:17 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 12 02:23:18.516457 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:18.517494 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Oct 12 02:23:18.517886 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:19.039749 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-e781c9cf-a892-4e02-8474-c80f158ca64a admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:23:19.058390 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-e781c9cf-a892-4e02-8474-c80f158ca64a admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.402985 Oct 12 02:23:19.058922 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 27/54] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 12 02:23:18 2022] POST /compute/v2.1/flavors => generated 404 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 12 02:23:20.274845 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:20.275845 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Oct 12 02:23:20.276242 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:20.797890 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-c4998a99-3311-493c-b6a7-f309e8cf53de admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:23:20.815864 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-c4998a99-3311-493c-b6a7-f309e8cf53de admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.402819 Oct 12 02:23:20.816404 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 28/56] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 12 02:23:20 2022] POST /compute/v2.1/flavors => generated 405 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:23:22.033646 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:22.034646 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001415 Oct 12 02:23:22.035041 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 29/57] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:22.557200 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-e2313410-93b8-43ba-8daf-1a59da32f6fe admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:23:22.577078 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-e2313410-93b8-43ba-8daf-1a59da32f6fe admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.398575 Oct 12 02:23:22.577606 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 29/58] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 12 02:23:22 2022] POST /compute/v2.1/flavors => generated 407 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:23:23.793220 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:23.794222 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Oct 12 02:23:23.794621 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 30/59] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:24.314374 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-aaa7d39a-1b02-4ced-93db-b47da6ebd904 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:23:24.332833 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-aaa7d39a-1b02-4ced-93db-b47da6ebd904 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.399525 Oct 12 02:23:24.333336 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 30/60] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 12 02:23:23 2022] POST /compute/v2.1/flavors => generated 409 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:23:25.542601 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:25.543638 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Oct 12 02:23:25.544040 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 31/61] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:23:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:23:26.058831 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-a44711e2-b894-4494-b430-95fbc1acd593 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:23:26.079226 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-a44711e2-b894-4494-b430-95fbc1acd593 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.399922 Oct 12 02:23:26.079756 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 31/62] 10.222.0.23 () {60 vars in 1202 bytes} [Wed Oct 12 02:23:25 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:24:21.988620 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:24:21.989945 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-e256b0e6-4587-4c5a-b55a-14a443885373 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001794 Oct 12 02:24:21.990346 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 32/63] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 12 02:24:21 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:24:22.001175 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:24:22.384847 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-50fe2cc3-d4e4-48c1-848b-74af25a03f82 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:24:22.404841 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-50fe2cc3-d4e4-48c1-848b-74af25a03f82 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.408024 Oct 12 02:24:22.405382 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 32/64] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 12 02:24:21 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:24:22.426281 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_api:SELECT=14 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:24:22.430595 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-c9db530b-e919-44fb-81a5-e3060b992e91 admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:24:22.438113 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-c9db530b-e919-44fb-81a5-e3060b992e91 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013289 Oct 12 02:24:22.438625 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 33/65] 10.222.0.23 () {60 vars in 1283 bytes} [Wed Oct 12 02:24:22 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:24:22.450567 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-482e852c-b296-428e-8310-c562a55a6468 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:24:22.457707 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-482e852c-b296-428e-8310-c562a55a6468 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010656 Oct 12 02:24:22.458220 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 33/66] 10.222.0.23 () {60 vars in 1286 bytes} [Wed Oct 12 02:24:22 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:24:22.469656 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:24:22.476806 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010514 Oct 12 02:24:22.477315 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 34/67] 10.222.0.23 () {60 vars in 1286 bytes} [Wed Oct 12 02:24:22 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 11 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:24:22.488090 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-c12f03b9-f32a-4f24-b52d-2d3b8e7238ed admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:24:22.495829 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-c12f03b9-f32a-4f24-b52d-2d3b8e7238ed admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011106 Oct 12 02:24:22.496330 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 34/68] 10.222.0.23 () {60 vars in 1283 bytes} [Wed Oct 12 02:24:22 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:27:23.092952 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:23.093962 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001460 Oct 12 02:27:23.094363 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 35/69] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:27:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:27:23.236206 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_api:SELECT=6 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:27:23.396095 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:23.400683 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:23.401219 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:23.401643 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:23.402350 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:23.402796 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:23.403210 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:23.434741 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-99593505-e684-4757-9991-cb6f62fdc5fd admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.92 time: 0.199708 Oct 12 02:27:23.435282 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 35/70] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 12 02:27:23 2022] GET /compute/v2.1/os-services => generated 794 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:27:24.505759 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:24.506803 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001550 Oct 12 02:27:24.507798 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 36/71] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:27:24 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:27:24.695164 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:24.698633 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:24.699117 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:24.699547 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:24.700294 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:24.700780 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:24.701206 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:24.720627 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-911183a1-681e-40c2-ac72-3a2a80aef26f admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.92 time: 0.074746 Oct 12 02:27:24.721191 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 36/72] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 12 02:27:24 2022] GET /compute/v2.1/os-services => generated 794 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:27:36.171822 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:36.172848 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 Oct 12 02:27:36.173238 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 37/73] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:27:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:27:36.320298 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_cell0:SELECT=4 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:27:36.321042 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_cell1:SELECT=4 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:27:36.321684 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG dbcounter [-] [103654] Writing DB stats nova_api:SELECT=8 {{(pid=103654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:27:36.369263 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:36.372603 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:36.373113 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:36.373547 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:36.374267 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:36.374726 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:36.375147 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:36.400067 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-b7ebf147-fbde-4883-a112-ae16f265a8c9 admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 1014 microversion: 2.92 time: 0.084192 Oct 12 02:27:36.400607 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 37/74] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 12 02:27:36 2022] GET /compute/v2.1/os-services => generated 1014 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:27:37.475174 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG nova.api.openstack.wsgi [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] Calling method '>' {{(pid=103653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:37.476675 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: INFO nova.api.openstack.requestlog [None req-f29e7b58-8c1d-47d1-b755-0cdc6708735c admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001913 Oct 12 02:27:37.477092 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: [pid: 103653|app: 0|req: 38/75] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 12 02:27:37 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:27:37.668049 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG nova.api.openstack.wsgi [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] Calling method '>' {{(pid=103654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:27:37.671398 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:37.671901 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:37.672356 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:37.673086 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:27:37.673527 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:27:37.673938 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: DEBUG oslo_concurrency.lockutils [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:27:37.695394 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: INFO nova.api.openstack.requestlog [None req-f230db0d-cfd0-4fca-b7a6-14cdf1a2dbe3 admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 1014 microversion: 2.92 time: 0.077603 Oct 12 02:27:37.695930 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: [pid: 103654|app: 0|req: 38/76] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 12 02:27:37 2022] GET /compute/v2.1/os-services => generated 1014 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:28:34.769145 dv-cinder-849143-14-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 12 02:28:34.769910 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: Wed Oct 12 02:28:34 2022 - graceful shutdown triggered... Oct 12 02:28:34.770354 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: Gracefully killing worker 1 (pid: 103653)... Oct 12 02:28:34.770719 dv-cinder-849143-14-iscsi devstack@n-api.service[103654]: Gracefully killing worker 2 (pid: 103654)... Oct 12 02:28:34.895034 dv-cinder-849143-14-iscsi devstack@n-api.service[103653]: DEBUG dbcounter [-] [103653] Writing DB stats nova_api:SELECT=4 {{(pid=103653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:28:35.769702 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: worker 1 buried after 1 seconds Oct 12 02:28:35.769928 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: worker 2 buried after 1 seconds Oct 12 02:28:35.769928 dv-cinder-849143-14-iscsi devstack@n-api.service[103652]: goodbye to uWSGI. Oct 12 02:28:35.771514 dv-cinder-849143-14-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 12 02:28:35.772048 dv-cinder-849143-14-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 12 02:28:35.774902 dv-cinder-849143-14-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 12 02:28:35.786540 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 12 02:28:35 2022] *** Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: nodename: dv-cinder-849143-14-iscsi Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: machine: x86_64 Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: clock source: unix Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: pcre jit disabled Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: detected number of CPU cores: 4 Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: current working directory: / Oct 12 02:28:35.788993 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: detected binary path: /usr/bin/uwsgi-core Oct 12 02:28:35.789654 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: your processes number limit is 47680 Oct 12 02:28:35.789654 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: your memory page size is 4096 bytes Oct 12 02:28:35.789654 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: detected max file descriptor number: 2048 Oct 12 02:28:35.789654 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: lock engine: pthread robust mutexes Oct 12 02:28:35.789654 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: thunder lock: enabled Oct 12 02:28:35.789654 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 12 02:28:35.789654 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 12 02:28:35.832735 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: Python main interpreter initialized at 0x559a3f5069c0 Oct 12 02:28:35.832735 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: python threads support enabled Oct 12 02:28:35.832735 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: your server socket listen backlog is limited to 100 connections Oct 12 02:28:35.832735 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: your mercy for graceful operations on workers is 90 seconds Oct 12 02:28:35.833178 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: mapped 403077 bytes (393 KB) for 2 cores Oct 12 02:28:35.833178 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: *** Operational MODE: preforking *** Oct 12 02:28:35.833444 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: *** uWSGI is running in multiple interpreter mode *** Oct 12 02:28:35.833444 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: spawned uWSGI master process (pid: 118841) Oct 12 02:28:35.833653 dv-cinder-849143-14-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 12 02:28:35.834468 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: spawned uWSGI worker 1 (pid: 118864, cores: 1) Oct 12 02:28:35.834593 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: spawned uWSGI worker 2 (pid: 118865, cores: 1) Oct 12 02:28:35.834760 dv-cinder-849143-14-iscsi devstack@n-api.service[118841]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 12 02:28:36.496986 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 12 02:28:36.504444 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 12 02:28:37.656986 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:28:37.656986 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: from cryptography.utils import int_from_bytes Oct 12 02:28:37.657577 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:28:37.657577 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: from cryptography.utils import int_from_bytes Oct 12 02:28:37.657929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:28:37.657929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: from cryptography.utils import int_from_bytes Oct 12 02:28:37.658364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 02:28:37.658364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: from cryptography.utils import int_from_bytes Oct 12 02:28:40.066880 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 02:28:40.068004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 02:28:40.068357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 02:28:40.068446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 02:28:40.068543 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 02:28:40.068626 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 02:28:40.068731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 02:28:40.069346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 02:28:40.069470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 02:28:40.069547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 02:28:40.069624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.069704 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.069787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.069857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.069934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070022 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070315 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070396 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070473 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070800 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.070874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.071235 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.071411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.071566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.071720 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.071866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.072016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.072189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849143-14-iscsi {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.072345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849143-14-iscsi {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.072496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.072761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.072937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.073091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.073246 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.073422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.073585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.073737 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.073890 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.074036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.074190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.074341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.074500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.074649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.074793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.074971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.075467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.075467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.075785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.076138 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.076528 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.076762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.076989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.077214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.077449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.077669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.077887 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.078106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.078312 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.078535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.078750 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.078969 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.079186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.079437 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849143-14-iscsi {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.079682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.079900 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.080115 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.080525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.080682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.080879 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.081031 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849143-14-iscsi {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.081187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.081796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.081960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.082121 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.082436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.082603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.082757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.082902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083054 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083467 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083550 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.083913 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084087 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084406 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084675 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.084957 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.085044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.085131 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.085222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.085222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.085471 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086293 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086376 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086464 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086803 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086891 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086983 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.086983 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087321 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087539 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.087987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.088078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.088312 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.088413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.088638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.088774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.088973 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.089069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.089282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.089385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.089590 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.089690 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.089897 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.089993 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.090207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.090308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.090516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.090606 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.090825 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.090920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.091134 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.091231 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.091463 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.091605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.091780 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.091885 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.092191 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.092296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.092416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.092516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.092767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.092914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.093102 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.093219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.093416 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.093521 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.093725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.093844 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.094049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.094150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.094361 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.094453 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.094670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.094774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.094980 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.095084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.095303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.095402 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.095615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.095726 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.095925 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.096034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.096180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.096276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.096490 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.096595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.096824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.096934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.097149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.097252 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.097423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.097527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.097778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.097888 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.098028 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.098124 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.098349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099473 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099627 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099695 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.099910 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.100119 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.100226 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.100424 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.100525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.100747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.100858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.101053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.101145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.101999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.102160 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.102282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.102397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.102528 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.102642 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.102922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.103065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.103189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.104515 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.104674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.104861 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.105024 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.105172 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.105322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.105468 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.105601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.105748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.105892 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106025 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106447 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106581 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106721 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106854 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.106985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107257 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107389 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107792 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.107990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108081 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108430 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.108785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.109818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.109959 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110210 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110312 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.110982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111158 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111244 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111733 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.111883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.112005 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.112234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.112356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.112478 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.112610 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.112945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.113237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.113362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.113362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.113574 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.113702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.113818 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.113930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 02:28:40.114056 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.114179 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.114305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.114430 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.114552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.114674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.114795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.114910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.115035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.115149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.115273 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.115273 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.115515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.115515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.116058 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.116200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.116333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.116463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.116604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.116763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.116934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.117071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.117202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.117333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.117473 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.117625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.117767 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.117934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.118075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.118759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.118852 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.118943 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119043 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119301 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119382 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119558 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.119959 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.120100 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.120195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.120429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.120524 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.120776 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.120873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.121092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.121198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.121425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.121517 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.121757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.121851 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.122077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.122168 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.122395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.122491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.122712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.122805 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.123029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.123131 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.123347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.123451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.123659 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.123760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.123977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.124082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.124294 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.124388 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.124622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.124811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.124952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.125055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.125266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.125357 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.125580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.125679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.125894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.126000 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.126205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.126303 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.126518 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.126609 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.126834 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.127007 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.127148 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.127250 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.127471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.127563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.127788 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.127881 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.128107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.128198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.128459 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.128609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.128834 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.128936 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.129060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.129153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.129368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.129461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.129677 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.129778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.129987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.130091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.130292 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.130393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.130598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.130698 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.130904 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.131003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.131216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.131308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.131523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.131616 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.131849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.131942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.132176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.132281 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.132500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.132642 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.132851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.132945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.133167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.133273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.133482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.133596 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.133794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.133889 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.134108 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.134219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.134414 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.134506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.134726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.134826 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.135041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.135144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.135349 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.135440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.135674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.135765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.135984 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.136127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.136290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.136425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.136605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.136718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.136937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.137031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.137247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.137339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.137560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.137659 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.137871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.137971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.138185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.138277 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.138569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.138726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.138870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.138959 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.139101 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.139200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.139420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.139529 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.139733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.139826 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.140065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.140192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.140397 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.140514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.140716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.140821 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.141052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.141154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.141381 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.141495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.141696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.141792 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.142000 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.142087 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.142304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.142400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.142618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.142728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.142920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.143015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.143223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.143313 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.143527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.143615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.143831 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.143928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.144134 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.144224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.144435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.144557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.144759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.144862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.145065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.145164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.145400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.145489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.145690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.145787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.145996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.146086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.146304 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.146393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.146609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.146709 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.146956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.147045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.147264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.147362 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.147609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.147708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.147925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.148022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.148799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.148955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.149115 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.149272 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.149431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.149534 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.149745 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.149837 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.150056 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.150171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.150374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.150479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.150686 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.150784 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.150996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.151090 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.151308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.151399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.151622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.151714 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.151939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.152031 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.152249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.152341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.152565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.152664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.152908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.153005 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.153236 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.153337 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.153545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.153660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.153866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.153971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.154201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.154299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.154518 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.154625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.154839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.154946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.155151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.155243 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.155462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.155590 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.155775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.155867 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.156084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.156175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.156397 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.156572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.156723 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.156817 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.157039 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.157130 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.157351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.157455 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.157660 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.157753 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.157975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.158072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.158285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.158386 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.158592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.158691 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.158908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.159051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.159226 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.159329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.159532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.159637 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.159849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.159942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.160165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.160256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.160493 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.160598 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.160824 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.160920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.161137 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.161238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.161453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.161545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.161762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.161853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.162070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.162169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.162380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.162477 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.162690 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.162798 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.163004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.163098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.163316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.163417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.163624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.163728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.164478 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.164582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.164815 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.164940 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.165127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.165267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.165434 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.165543 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.165741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.165848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.166047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.166157 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.166355 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.166465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.166669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.166777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.166991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.167097 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.167301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.167400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.167610 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.167716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.167934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.168039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.168241 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.168340 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.168547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.168654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.168875 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.168984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.169189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.169347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.169574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.169760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.169884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.169974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.170066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.170152 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.170427 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.170677 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.170771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.170863 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.171018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.171120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.171339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.171442 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.171665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.171765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.171978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.172069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.172292 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.172392 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.172606 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.172729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.172980 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.173131 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.173248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.173394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.173544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.173643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.173859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.173952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.174172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.174272 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.174488 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.174589 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.174800 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.174893 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.175113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.175350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.175456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.175541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.175720 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.175830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.176037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.176138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.176348 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.176446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.176653 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.176964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.177061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.177296 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.177397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.177614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.177874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.177982 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.178211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.178304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.178534 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.178669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.178857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.179002 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.179309 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.179486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.179677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.179814 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.179915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.180028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.180188 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.180278 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.180552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.180677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.180876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.181104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.181263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.181408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.181561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.181706 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.181857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.182012 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.182164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.182309 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.182452 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.182595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.182738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.182900 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.183071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.183209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.183353 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.183504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.183647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.183799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.183950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.184102 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.184243 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.184394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.184536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.184754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.184853 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.184951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.185038 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.185123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.185337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.185487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.185627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.185750 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.185953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.186084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.186260 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.186347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.186560 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.186666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.186867 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.186961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.187174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.187270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.187522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.187616 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.187828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.187921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.188171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.188264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.188482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.188583 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.188833 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.188949 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.189144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.189251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.189455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.189561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.189761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.189879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.190070 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.190167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.190376 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.190474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.190683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.190799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.190992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.191090 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.191302 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.191401 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.191605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.191707 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.191917 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.192008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.192222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.192311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.192532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.192623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.192859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.192955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.193207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.193305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.193486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.193705 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.193851 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.194008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.194161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.194311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.194468 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.194633 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.194777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.194920 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.195088 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.195235 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.195389 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.195546 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.195689 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.195833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.196038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.196190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.196362 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.196514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.196681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.196874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.197031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.197186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.197331 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.197474 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.197624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.197769 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.197918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.198067 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.198220 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.198360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.198504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.198664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.198809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.198960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.199111 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.199261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.199409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.199707 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.199802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.199896 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.199989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.200082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.200278 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.200417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.200583 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.200678 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.200972 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.201144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.201232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.201330 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.201523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.201618 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.201830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.201923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.202139 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.202242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.202451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.202542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.202758 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.202858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.203063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.203164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.203367 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.203460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.203675 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.203773 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.203980 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.204082 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.204284 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.204377 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.204591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.204684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.204924 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.205021 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.205230 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.205333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.205539 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.205638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.205848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.205946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.206152 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.206263 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.206460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.206561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.206766 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.206862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.207118 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.207212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.207427 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.207555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.207762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.207854 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.208080 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.208172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.208396 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.208503 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.208732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.208840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.209044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.209137 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.209358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.209463 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.209699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.209806 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.209973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.210064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.210280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.210429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.210590 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.211296 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.211479 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.211573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.211786 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.211889 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.212094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.212196 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.212417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.212519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.212748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.212847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.213066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.213169 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.213393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.213485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.213706 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.213809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.214024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.214117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.214328 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.214422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.214639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.214734 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.214954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.215047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.215266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.215365 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.215581 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.215683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.215895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.215991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.216209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.216321 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.216521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.216616 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.216852 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.216984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.217167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.217301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.217483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.217586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.217790 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.217882 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.218103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.218252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.218420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.218515 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.218729 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.218831 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.219042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.219144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.219353 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.219446 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.219660 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.219767 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.219969 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.220074 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.220273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.220409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.220586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.220749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.220917 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.221071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.221355 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.221470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.221671 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.221766 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.221985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.222088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.222295 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.222397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.222608 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.222701 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.222920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.223057 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.223237 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.223368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.223562 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.223657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.223901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.224003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.224259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.224360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.224567 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.224673 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.224902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.225008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.225218 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.225311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.225523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.225624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.225830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.225928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.226142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.226265 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.226990 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.227098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.227305 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.227409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.227614 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.227708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.227922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.228025 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.228248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.228342 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.228555 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.228649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.228905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.229004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.229327 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.229423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.229642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.229730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.229955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.230065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.230305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.230399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.230630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.230734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.230959 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.231058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.231270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.231379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.231593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.231720 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.231909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.232018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.232216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.232319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.232534 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.232644 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.232869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.232984 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.233195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.233299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.233503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.233620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.233825 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.233927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.234141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.234248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.234451 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.234546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.234777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.234869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.235100 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.235209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.235419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.235513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.235762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.235853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.236133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.236261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.236472 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.236645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.236827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.236993 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.237159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.237316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.237465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.237613 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.237759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.237903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.238061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.238227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.238386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.238539 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.238684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.238828 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.239050 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.239202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.239351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.239496 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.239642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.239794 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.239945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.240125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.240326 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.240422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.240507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.240592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.241012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.241153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.241278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.241395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.241623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.241740 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.241945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.242065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.242272 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.242367 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.242592 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.242700 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.242921 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.243023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.243253 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.243347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.243563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.243667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.243876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.243981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.244187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.244290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.244496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.244598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.244826 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.244939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.245138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.245232 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.245487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.245583 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.245795 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.245904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.246122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.246286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.246462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.246553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.246772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.246864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.247084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.247176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.247397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.247495 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.247719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.247820 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.248045 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.248151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.248350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.248491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.248657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.248834 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.248997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.249140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.249306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.249438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.249613 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.249712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.249919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.250022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.250224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.250355 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.250528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.250624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.250832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.250934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.251137 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.251238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.251440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.251535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.251751 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.251877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.252058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.252172 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.252373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.252475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.252677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.252799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.253014 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.253122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.253321 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.253420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.253632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.253730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.253951 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.254043 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.254243 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.254344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.254579 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.254678 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.254883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.254983 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.255214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.255333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.255523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.255614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.255840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.255941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.256161 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.256274 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.256479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.256658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.256855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.256959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.257128 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.257233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.257444 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.257565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.257786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.257880 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.258102 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.258207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.258416 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.258517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.258735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.258968 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.259062 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.259304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.259400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.259626 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.259892 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.259987 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.260228 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.260373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.260549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.260648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.260893 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.260990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.261209 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.261304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.261522 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.261615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.261839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.261941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.262155 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.262279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.262468 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.262575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.262780 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.262889 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.263093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.263187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.263427 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.263528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.263756 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.263865 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.264091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.264200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.264395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.264497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.264732 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.264894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.265037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.265182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.265354 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.265495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.265672 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.265816 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.266002 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.266148 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.266333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.266505 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.266628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.266723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.266941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.267036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.267251 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.267354 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.267564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.267673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.267877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.267984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.268186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.268293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.268497 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.268591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.268831 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.268929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.269147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.269257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.269457 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.269625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.269769 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.269934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.270084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.270188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.270411 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.270505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.270727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.270829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.271050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.271153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.271366 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.271576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.271677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.271895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.271990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.272237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.272493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.272599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.272833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.272956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.273689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.273809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.274017 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.274118 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.274347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.274448 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.274670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.274768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.274982 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.275073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.275302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.275407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.275610 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.275698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.275917 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.276015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.276230 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.276340 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.276543 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.276652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.276874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.276981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.277197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.277295 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.277508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.277792 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.277883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.278049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.278226 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.278401 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.278554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.278721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.278940 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.279092 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.279289 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.279446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.279599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.279750 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.279922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.280076 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.280251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.280405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.280562 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.280752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.280891 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.281005 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.281213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.281360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.281520 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.281614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.281825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.281928 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.282135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.282234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.282451 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.282571 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.282760 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.282877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.283065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.283168 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.283378 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.283470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.283688 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.283792 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.283991 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.284083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.284320 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.284476 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.284613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.284740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.284948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.285044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.285271 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.285387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.285572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.285671 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.285876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.285970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.286183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.286284 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.286494 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.286602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.286803 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.286915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.287106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.287216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.287409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.287518 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.287723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.287824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.288026 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.288155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.288421 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.288529 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.288746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.288893 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.289619 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.289728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.289945 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.290042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.290253 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.290351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.290563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.290747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.290899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.291054 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.291209 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.291354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.291517 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.291657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.291819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.291970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.292116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.292257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.292415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.292595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.292752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.292992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.293136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.293283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.293443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.293613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.293774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.293936 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.294161 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.294305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.294497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.294652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.294825 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.294993 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.295165 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.295333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.295491 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.295634 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.295775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.295927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.296093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.296245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.296399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.296565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.296759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.296870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.297016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.297125 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.297330 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.297425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.297641 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.297742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.297953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.298055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.298268 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.298381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.298575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.298677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.298884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.298987 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.299195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.299292 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.299504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.299594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.299813 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.299914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.300124 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.300231 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.300433 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.300525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.300779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.300892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.301080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.301183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.301395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.301497 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.301728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.301818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.302048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.302155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.302360 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.302468 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.302673 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.302772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.302990 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.303075 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.303299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.303407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.303612 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.303711 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.303927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.304026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.304238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.304414 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.304572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.304773 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.304929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.305033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.305244 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.305339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.305552 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.305655 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.305942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.306041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.306256 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.306354 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.306576 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.306670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.306899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.307011 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.307210 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.307309 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.307529 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.307622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.307845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.307950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.308156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.308257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.308479 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.308594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.308809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.308924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.309142 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.309238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.309465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.309566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.309774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.309869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.310082 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.310183 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.310395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.310502 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.310699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.310808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.311009 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.311119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.311311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.311426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.311616 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.311725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.311922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.312031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.312227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.312330 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.312527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.312635 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.312858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.312952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.313162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.313259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.313466 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.313568 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.313768 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.313865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.314071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.314170 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.314373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.314470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.314677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.314848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.315005 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.315159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.315306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.315457 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.315602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.315758 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.315922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.316075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.316221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.316373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.316522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.316664 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.316796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.316979 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.317077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.317173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.317321 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.317425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.317625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.317722 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.317929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.318023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.318261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.318374 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.318611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.318728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.318904 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.319003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.319216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.319336 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.319524 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.319619 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.319848 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.319944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.320187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.320294 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.320500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.320601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.320846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.320962 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.321161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.321283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.321475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.321584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.321784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.321876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.322100 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.322215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.322417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.322536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.322729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.322832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.323040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.323148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.323352 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.323453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.323666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.323776 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.323992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.324148 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.324315 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.324411 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.324621 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.324765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.324969 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.325075 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.325283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.325377 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.325601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.325709 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.325915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.326008 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.326226 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.326320 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.326536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.326650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.326852 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.326999 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.327162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.327264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.327472 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.327580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.327784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.327877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.328092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.328193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.328407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.328510 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.328738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.328871 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.329055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.329157 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.329371 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.329481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.329681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.329780 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.330006 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.330114 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.330332 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.330432 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.330641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.330742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.330966 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.331090 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.331291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.331407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.331630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.331730 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.331941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.332052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.332256 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.332362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.332564 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.332664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.332830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 02:28:40.332938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.333228 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.333533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 02:28:40.333781 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 02:28:40.395031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO dbcounter [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Registered counter for database nova_api Oct 12 02:28:40.395664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO dbcounter [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Registered counter for database nova_api Oct 12 02:28:40.415950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_db.sqlalchemy.engines [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.417575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_db.sqlalchemy.engines [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.422757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writer thread running {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.424345 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writer thread running {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.479404 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.context [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dbde3dce-57ec-471e-b3b0-530d5203c2f2(cell1) {{(pid=118864) load_cells /opt/stack/nova/nova/context.py:464}} Oct 12 02:28:40.481072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.context [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dbde3dce-57ec-471e-b3b0-530d5203c2f2(cell1) {{(pid=118865) load_cells /opt/stack/nova/nova/context.py:464}} Oct 12 02:28:40.483505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:28:40.484098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:28:40.485093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:28:40.485506 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:28:40.485628 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:28:40.486249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:28:40.486698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:28:40.487108 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:28:40.487838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:28:40.488280 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:28:40.488718 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:28:40.490291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:28:40.490981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO dbcounter [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Registered counter for database nova_cell0 Oct 12 02:28:40.492564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO dbcounter [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Registered counter for database nova_cell0 Oct 12 02:28:40.495117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO dbcounter [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Registered counter for database nova_cell1 Oct 12 02:28:40.496871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO dbcounter [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Registered counter for database nova_cell1 Oct 12 02:28:40.502302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_db.sqlalchemy.engines [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.504586 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_db.sqlalchemy.engines [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.509445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writer thread running {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.512261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_db.sqlalchemy.engines [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.513075 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_db.sqlalchemy.engines [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.520941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writer thread running {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.521441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writer thread running {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.654190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writer thread running {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.663112 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO dbcounter [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Registered counter for database nova_cell0 Oct 12 02:28:40.669835 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO dbcounter [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Registered counter for database nova_cell0 Oct 12 02:28:40.670018 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_db.sqlalchemy.engines [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.674827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writer thread running {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.680221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_db.sqlalchemy.engines [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 02:28:40.684987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writer thread running {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 02:28:40.995688 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: WARNING keystonemiddleware.auth_token [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:28:41.002999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: WARNING keystonemiddleware.auth_token [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:28:41.022426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: WARNING keystonemiddleware.auth_token [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:28:41.029946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: WARNING keystonemiddleware.auth_token [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:28:41.344639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: WARNING keystonemiddleware.auth_token [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:28:41.350588 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: WARNING keystonemiddleware.auth_token [None req-8bca4e08-d8b6-468f-b6b5-310f8830cc26 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:28:41.356384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559a3f5069c0 pid: 118864 (default app) Oct 12 02:28:41.378047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: WARNING keystonemiddleware.auth_token [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 12 02:28:41.384939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: WARNING keystonemiddleware.auth_token [None req-32cb0efd-cef8-45e4-b431-bf5477d9cef8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 12 02:28:41.391482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559a3f5069c0 pid: 118865 (default app) Oct 12 02:29:16.701098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:16.702254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:16.703438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:16.704678 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:18.247519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-82398f7f-f71d-4ca3-8c2f-e7c3b44d1754 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Action: 'create', calling method: .version_select at 0x7f59d45c0550>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-274897311", "metadata": {"Type": "work"}}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-82398f7f-f71d-4ca3-8c2f-e7c3b44d1754 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Request method failure captured: Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 121 Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiZ86YbHAJevmCsxgkhJypCiI1iOpDaShCgwz2aTfV_Fcwbh7WX2pLiggYJCLZakGijFJWA5TfHGoy6CDpGC5Q-QQ5_jGC8pqPDKrspCh7fXsYvCRGh8pNEXZE0QHU8KiDCQZye5Y82WW4i-wtoZSWx0AbrrCrtqhkrLJ05cSVF5qJY Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:29:18.255783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_2d2ef7ec625d4fd79045b3d5001dc790", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: f428a9f1b0834b89b4a87e48469d224a Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-274897311", "metadata": {"Type": "work"}}} Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: Oct 12 02:29:18.257811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-274897311', 'metadata': {'Type': 'work'}}}} Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise error Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi '#$%' Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:29:18.258808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 12 02:29:18.261632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 12 02:29:18.263527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 12 02:29:18.263527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.263527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-82398f7f-f71d-4ca3-8c2f-e7c3b44d1754 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:29:18.263527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-82398f7f-f71d-4ca3-8c2f-e7c3b44d1754 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.571359 Oct 12 02:29:18.263527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 1/1] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 12 02:29:16 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1578 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:29:18.845694 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3d4664ce-1aef-46b8-9809-312c56ea22cb tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Action: 'create', calling method: .version_select at 0x7f59d464b0d0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-764193480", "metadata": {"Type": "work"}}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3d4664ce-1aef-46b8-9809-312c56ea22cb tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Request method failure captured: Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 119 Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiZ86YbHAJevmCsxgkhJypCiI1iOpDaShCgwz2aTfV_Fcwbh7WX2pLiggYJCLZakGijFJWA5TfHGoy6CDpGC5Q-QQ5_jGC8pqPDKrspCh7fXsYvCRGh8pNEXZE0QHU8KiDCQZye5Y82WW4i-wtoZSWx0AbrrCrtqhkrLJ05cSVF5qJY Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:29:18.873626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_2d2ef7ec625d4fd79045b3d5001dc790", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: f428a9f1b0834b89b4a87e48469d224a Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-764193480", "metadata": {"Type": "work"}}} Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: Oct 12 02:29:18.874916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-764193480', 'metadata': {'Type': 'work'}}}} Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise error Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi '0' Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:29:18.875661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 12 02:29:18.876992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 12 02:29:18.878080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 12 02:29:18.878080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.878080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3d4664ce-1aef-46b8-9809-312c56ea22cb tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:29:18.878080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3d4664ce-1aef-46b8-9809-312c56ea22cb tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.607623 Oct 12 02:29:18.878080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 1/2] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 12 02:29:18 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 609 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:29:18.886361 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-95516c57-a927-4697-953e-25473b59a682 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Action: 'create', calling method: .version_select at 0x7f59d45c0e50>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-828668083", "metadata": {"Type": "work"}}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-95516c57-a927-4697-953e-25473b59a682 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Request method failure captured: Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 118 Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiZ86YbHAJevmCsxgkhJypCiI1iOpDaShCgwz2aTfV_Fcwbh7WX2pLiggYJCLZakGijFJWA5TfHGoy6CDpGC5Q-QQ5_jGC8pqPDKrspCh7fXsYvCRGh8pNEXZE0QHU8KiDCQZye5Y82WW4i-wtoZSWx0AbrrCrtqhkrLJ05cSVF5qJY Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:29:18.891063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_2d2ef7ec625d4fd79045b3d5001dc790", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: f428a9f1b0834b89b4a87e48469d224a Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-828668083", "metadata": {"Type": "work"}}} Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: Oct 12 02:29:18.892475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-828668083', 'metadata': {'Type': 'work'}}}} Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise error Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi '' Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:29:18.893258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 12 02:29:18.894504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 12 02:29:18.895598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 12 02:29:18.895598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:18.895598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-95516c57-a927-4697-953e-25473b59a682 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:29:18.895598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-95516c57-a927-4697-953e-25473b59a682 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.012034 Oct 12 02:29:18.895598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 2/3] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 12 02:29:18 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:29:18.901645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:18.902493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:18.903200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:18.909813 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:18.921885 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5e6d3f92-152c-4740-a493-2453d0f5b872 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Calling method '.version_select at 0x7f59d45ffaf0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:20.359617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-5e6d3f92-152c-4740-a493-2453d0f5b872 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] HTTP exception thrown: Volume tempest-invalid-1848494049 could not be found. Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5e6d3f92-152c-4740-a493-2453d0f5b872 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Request method failure captured: Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1848494049 HTTP/1.1 Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiZ86YbHAJevmCsxgkhJypCiI1iOpDaShCgwz2aTfV_Fcwbh7WX2pLiggYJCLZakGijFJWA5TfHGoy6CDpGC5Q-QQ5_jGC8pqPDKrspCh7fXsYvCRGh8pNEXZE0QHU8KiDCQZye5Y82WW4i-wtoZSWx0AbrrCrtqhkrLJ05cSVF5qJY Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:29:20.365129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_2d2ef7ec625d4fd79045b3d5001dc790", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: f428a9f1b0834b89b4a87e48469d224a Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Volume tempest-invalid-1848494049 could not be found. Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'tempest-invalid-1848494049'} Oct 12 02:29:20.366723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1848494049 could not be found. (HTTP 404) (Request-ID: req-61f4426d-ef62-4a12-8ec4-eaab634c317e) Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.367489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 12 02:29:20.368709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1848494049 could not be found. Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:29:20.369852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1848494049 could not be found. Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1848494049 could not be found. Oct 12 02:29:20.370995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.373702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5e6d3f92-152c-4740-a493-2453d0f5b872 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Returning 404 to user: Volume tempest-invalid-1848494049 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:29:20.375857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5e6d3f92-152c-4740-a493-2453d0f5b872 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1848494049" status: 404 len: 99 microversion: 2.1 time: 1.475096 Oct 12 02:29:20.382125 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1218 bytes} [Wed Oct 12 02:29:18 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1848494049 => generated 99 bytes in 1482 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Oct 12 02:29:20.386566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9d763a35-26b7-4209-b60c-5a1272af4b5a tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003842 Oct 12 02:29:20.391034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 3/5] 10.222.0.23 () {58 vars in 1140 bytes} [Wed Oct 12 02:29:20 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 9 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 12 02:29:20.401833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-33fbbcae-84b8-48ad-bf44-aab93d711eae tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.009221 Oct 12 02:29:20.402617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 3/6] 10.222.0.23 () {58 vars in 1137 bytes} [Wed Oct 12 02:29:20 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 11 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Oct 12 02:29:20.415431 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e15bbe68-06a7-41cc-89ed-58d4eec28816 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Calling method '.version_select at 0x7f59d447d4c0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:20.666233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-e15bbe68-06a7-41cc-89ed-58d4eec28816 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] HTTP exception thrown: Volume 33f59d67-dc9e-4376-8842-5509f5c500b8 could not be found. Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e15bbe68-06a7-41cc-89ed-58d4eec28816 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Request method failure captured: Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/os-volumes/33f59d67-dc9e-4376-8842-5509f5c500b8 HTTP/1.1 Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiZ86YbHAJevmCsxgkhJypCiI1iOpDaShCgwz2aTfV_Fcwbh7WX2pLiggYJCLZakGijFJWA5TfHGoy6CDpGC5Q-QQ5_jGC8pqPDKrspCh7fXsYvCRGh8pNEXZE0QHU8KiDCQZye5Y82WW4i-wtoZSWx0AbrrCrtqhkrLJ05cSVF5qJY Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:29:20.670145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_2d2ef7ec625d4fd79045b3d5001dc790", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: f428a9f1b0834b89b4a87e48469d224a Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Volume 33f59d67-dc9e-4376-8842-5509f5c500b8 could not be found. Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '33f59d67-dc9e-4376-8842-5509f5c500b8'} Oct 12 02:29:20.671445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 33f59d67-dc9e-4376-8842-5509f5c500b8 could not be found. (HTTP 404) (Request-ID: req-3ec9b645-ce70-42d7-8e8e-60a0d1f167a6) Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.672286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 12 02:29:20.673501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 33f59d67-dc9e-4376-8842-5509f5c500b8 could not be found. Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 12 02:29:20.677086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 33f59d67-dc9e-4376-8842-5509f5c500b8 could not be found. Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 33f59d67-dc9e-4376-8842-5509f5c500b8 could not be found. Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e15bbe68-06a7-41cc-89ed-58d4eec28816 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Returning 404 to user: Volume 33f59d67-dc9e-4376-8842-5509f5c500b8 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e15bbe68-06a7-41cc-89ed-58d4eec28816 tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/33f59d67-dc9e-4376-8842-5509f5c500b8" status: 404 len: 109 microversion: 2.1 time: 0.262492 Oct 12 02:29:20.680307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:29:20 2022] DELETE /compute/v2.1/os-volumes/33f59d67-dc9e-4376-8842-5509f5c500b8 => generated 109 bytes in 264 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:29:20.682455 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9d46bf81-1720-4cca-a8da-e4280084180f tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Calling method '.version_select at 0x7f59d43a90d0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:20.714742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-9d46bf81-1720-4cca-a8da-e4280084180f tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] HTTP exception thrown: Volume bd47dfef-a3e5-4b5c-aece-04c7c953edac could not be found. Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9d46bf81-1720-4cca-a8da-e4280084180f tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Request method failure captured: Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/os-volumes/bd47dfef-a3e5-4b5c-aece-04c7c953edac HTTP/1.1 Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiZ86YbHAJevmCsxgkhJypCiI1iOpDaShCgwz2aTfV_Fcwbh7WX2pLiggYJCLZakGijFJWA5TfHGoy6CDpGC5Q-QQ5_jGC8pqPDKrspCh7fXsYvCRGh8pNEXZE0QHU8KiDCQZye5Y82WW4i-wtoZSWx0AbrrCrtqhkrLJ05cSVF5qJY Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:29:20.716616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2d2ef7ec625d4fd79045b3d5001dc790"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_2d2ef7ec625d4fd79045b3d5001dc790", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 2d2ef7ec625d4fd79045b3d5001dc790 Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-VolumesNegativeTest-181731744 Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: f428a9f1b0834b89b4a87e48469d224a Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-VolumesNegativeTest-181731744-project Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Volume bd47dfef-a3e5-4b5c-aece-04c7c953edac could not be found. Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'bd47dfef-a3e5-4b5c-aece-04c7c953edac'} Oct 12 02:29:20.717955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bd47dfef-a3e5-4b5c-aece-04c7c953edac could not be found. (HTTP 404) (Request-ID: req-f68c6df6-8b84-45da-a111-b5ae31a5d6be) Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.719150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:29:20.730398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bd47dfef-a3e5-4b5c-aece-04c7c953edac could not be found. Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 12 02:29:20.733384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume bd47dfef-a3e5-4b5c-aece-04c7c953edac could not be found. Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bd47dfef-a3e5-4b5c-aece-04c7c953edac could not be found. Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9d46bf81-1720-4cca-a8da-e4280084180f tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] Returning 404 to user: Volume bd47dfef-a3e5-4b5c-aece-04c7c953edac could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9d46bf81-1720-4cca-a8da-e4280084180f tempest-VolumesNegativeTest-181731744 tempest-VolumesNegativeTest-181731744-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/bd47dfef-a3e5-4b5c-aece-04c7c953edac" status: 404 len: 109 microversion: 2.1 time: 0.040526 Oct 12 02:29:20.735339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:29:20 2022] GET /compute/v2.1/os-volumes/bd47dfef-a3e5-4b5c-aece-04c7c953edac => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:29:31.887095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7228950f-b138-4c7e-83ec-26005999519d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'create', calling method: .version_select at 0x7f59d4403f70>, body: {"keypair": {"name": "tempest-keypair-2139911655", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE9NC9P478FkdPrKE1VlJxYIHUo4cIaQybscMbjBPG1QL6fHd/q3aBaw0gnWsEr2SAsq6JiM/glKb5nAD4gfyEOUd4zLqT6U4VyBbY9/BQdQNCfGd2f1QOhnUiwvZFDJ/g=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:31.964748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-7228950f-b138-4c7e-83ec-26005999519d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:29:31.975166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-7228950f-b138-4c7e-83ec-26005999519d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for user 43f16ca13925411fb76a330b6c95395d and project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:29:32.088095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7228950f-b138-4c7e-83ec-26005999519d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.735929 Oct 12 02:29:32.089091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 5/9] 10.222.0.23 () {62 vars in 1200 bytes} [Wed Oct 12 02:29:31 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 738 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Oct 12 02:29:33.685489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-165621514", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "networks": [{"uuid": "5c71fac5-abd5-4133-a9fd-cd2743c1f33e"}]}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:33.903449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] validate_networks() for [('5c71fac5-abd5-4133-a9fd-cd2743c1f33e', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:29:35.792500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:29:35.793142 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:29:35.793389 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:29:35.793790 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:29:35.794192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:29:35.834449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:35.838924 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:35.839458 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:35.840275 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:35.843303 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:35.843830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:35.952407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Getting quotas for project 60e8d0fb43eb419ba1c5eb830cc89ee3. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:29:35.992172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Getting quotas for user 47f30e76f6894e5f9c06f71adde020e6 and project 60e8d0fb43eb419ba1c5eb830cc89ee3. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:29:36.052373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:29:36.058265 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:29:36.192334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1ff8f965-8839-4bce-a0fd-ec88c35bdddf tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.738613 Oct 12 02:29:36.195620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 5/10] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:29:33 2022] POST /compute/v2.1/servers => generated 380 bytes in 2747 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:29:36.208098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7a1556b0-23d5-4c26-bd77-a7a6101f6e68 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:36.212936 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7a1556b0-23d5-4c26-bd77-a7a6101f6e68 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:36.364167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1727504812", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:36.615717 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] validate_networks() for [('d2208204-5dac-4ba9-9cf8-255af28eb635', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:29:36.805214 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7a1556b0-23d5-4c26-bd77-a7a6101f6e68 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:36.841673 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7a1556b0-23d5-4c26-bd77-a7a6101f6e68 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1214 microversion: 2.1 time: 0.636788 Oct 12 02:29:36.842587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 6/11] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:36 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1214 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:37.873254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c5dc7453-6989-4119-b8e1-04e81debfb8b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:37.874167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c5dc7453-6989-4119-b8e1-04e81debfb8b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:38.011953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c5dc7453-6989-4119-b8e1-04e81debfb8b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:38.021748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c5dc7453-6989-4119-b8e1-04e81debfb8b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1214 microversion: 2.1 time: 0.153602 Oct 12 02:29:38.023277 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 7/12] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:37 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1214 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:38.061512 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-2120414230", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8e5a7b02-9fc0-4dda-b1f4-8267371dc191"}], "security_groups": [{"name": "tempest-securitygroup--1327024506"}], "key_name": "tempest-keypair-2139911655", "user_data": "}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:38.519740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] validate_networks() for [('8e5a7b02-9fc0-4dda-b1f4-8267371dc191', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:29:39.494763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:29:39.495406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:29:39.495891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:29:39.496290 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:29:39.496693 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:29:39.527006 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:39.527636 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:39.528060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:39.532963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:39.533950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:39.534784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:39.589467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:29:39.612774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for user bcd7f1121dab4e8db4c747c51ed49240 and project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:29:39.645444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:29:39.649557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:29:39.825097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-14b349ba-ab5d-4819-aa73-aff820802021 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.047686 Oct 12 02:29:39.828207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 6/13] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:29:35 2022] POST /compute/v2.1/servers => generated 383 bytes in 4052 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:29:40.543853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-99a14e65-794f-4b7d-8c3e-c87367f6707b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: .version_select at 0x7f59d426f0d0>, body: {"keypair": {"name": "tempest-keypair-780906542", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDzDoXvwsTBccfrjeahqzzLr9+AYeYTWCq2xcjxs/S8nA23ITWWpJn44TOSTHNdSHDQog/g4g8Tzf7n+HT9nVXElr+E1Dxul1Riff8C7C0UgPPMmXdmbBLQCBU0cWYcg5g=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:40.639899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-99a14e65-794f-4b7d-8c3e-c87367f6707b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:29:40.655742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-99a14e65-794f-4b7d-8c3e-c87367f6707b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:29:40.769077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:29:40.771648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:29:40.772378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:29:40.773956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:29:40.774539 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:29:40.799728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-99a14e65-794f-4b7d-8c3e-c87367f6707b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.969736 Oct 12 02:29:40.821725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 7/14] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:29:39 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 993 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:29:40.832819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:40.833889 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:40.834335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:40.835136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:40.835586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:40.836006 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:40.846826 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:40.851064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:40.948130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:29:40.950552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:40.953380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for user 43f16ca13925411fb76a330b6c95395d and project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:29:40.956172 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:40.959609 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:40.962193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:40.962698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:40.966559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:40.989011 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:29:40.995520 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:29:41.187524 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-044b8749-28bb-4abd-8bdc-8d631fe04937 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 3.158537 Oct 12 02:29:41.188827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 8/15] 10.222.0.23 () {62 vars in 1188 bytes} [Wed Oct 12 02:29:38 2022] POST /compute/v2.1/servers => generated 402 bytes in 3165 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 12 02:29:41.222082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-813a0006-58cb-4c57-803d-e5f5d037366b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:41.229854 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-813a0006-58cb-4c57-803d-e5f5d037366b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:41.495046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-813a0006-58cb-4c57-803d-e5f5d037366b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:41.502902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-813a0006-58cb-4c57-803d-e5f5d037366b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1222 microversion: 2.1 time: 0.304388 Oct 12 02:29:41.506662 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 9/16] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:41 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1222 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:41.556196 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9cdeb4ac-f8ad-4b4c-8236-ab175cd1a6b1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:41.558398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9cdeb4ac-f8ad-4b4c-8236-ab175cd1a6b1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:41.697117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9cdeb4ac-f8ad-4b4c-8236-ab175cd1a6b1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:41.705503 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9cdeb4ac-f8ad-4b4c-8236-ab175cd1a6b1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:41.708825 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9cdeb4ac-f8ad-4b4c-8236-ab175cd1a6b1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:41.711010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9cdeb4ac-f8ad-4b4c-8236-ab175cd1a6b1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1282 microversion: 2.20 time: 0.175796 Oct 12 02:29:41.716201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 10/17] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:41 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1282 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:41.807122 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:41.847289 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1c5956ea-1318-43f4-b2d3-a5ad866f7d99 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1272 microversion: 2.1 time: 1.017304 Oct 12 02:29:41.847951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 8/18] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:40 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1272 bytes in 1026 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:42.233908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1046025114", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:42.425541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] validate_networks() for [('d2208204-5dac-4ba9-9cf8-255af28eb635', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:29:42.741459 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:42.742077 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:42.753395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:42.767000 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:42.767634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:42.769958 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:42.772353 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:42.772910 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:43.222487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:43.233022 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:43.234502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:43.235617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-61a0045f-bc20-4901-bd37-b28875ab3bb8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1286 microversion: 2.20 time: 0.498819 Oct 12 02:29:43.253745 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 9/19] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:42 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1286 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:29:43.264285 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:43.273914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:43.285806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:43.286531 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:43.286973 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:43.287830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:43.288300 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:43.288747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:43.412001 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:43.419997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-02c52884-adb8-42e8-88c7-cf56612a222b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1272 microversion: 2.1 time: 0.160667 Oct 12 02:29:43.420578 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 10/20] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:43 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1272 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:44.060584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:29:44.061104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:29:44.061693 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:29:44.065590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:29:44.066312 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:29:44.089883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:44.090601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:44.091247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:44.092187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:44.093959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:44.094614 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:44.143208 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:29:44.153968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for user bcd7f1121dab4e8db4c747c51ed49240 and project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:29:44.182914 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:29:44.191229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:29:44.278461 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:44.281385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:44.294095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:44.294880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:44.295502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:44.296389 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:44.297051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:44.299161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:44.300347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b5a8c6e9-1ae3-4aa1-9013-b94015d1cb83 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.070819 Oct 12 02:29:44.305961 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 11/21] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:29:42 2022] POST /compute/v2.1/servers => generated 383 bytes in 2077 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:29:44.317182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5b69a1b7-ec18-41b8-9959-6970cba77002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:44.323058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5b69a1b7-ec18-41b8-9959-6970cba77002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:44.443283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5b69a1b7-ec18-41b8-9959-6970cba77002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:44.453707 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5b69a1b7-ec18-41b8-9959-6970cba77002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1222 microversion: 2.1 time: 0.143557 Oct 12 02:29:44.462563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 12/22] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:44 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1222 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:44.480260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:44.486414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:44.513902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:44.515574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:44.520190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:44.523115 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:44.526680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:44.527551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:44.534846 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:44.549711 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:44.556092 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:44.563771 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8c783f8f-8874-48ff-94e1-33faa7d6786b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1286 microversion: 2.20 time: 0.289038 Oct 12 02:29:44.568944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 11/23] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:44 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1286 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:45.421211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:45.442880 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d2a7e2d0-6f11-48ff-91dc-9a26e1b5816b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.978396 Oct 12 02:29:45.465485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 13/24] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:44 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 1002 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:45.479449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cb7bcbb2-2869-4f8f-8248-d2669b2d5874 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:45.480067 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cb7bcbb2-2869-4f8f-8248-d2669b2d5874 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:45.609082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:45.609968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:45.622523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:45.623027 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:45.623469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:45.624178 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:45.633207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:45.633548 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:45.887141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:45.899796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:45.901811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cb7bcbb2-2869-4f8f-8248-d2669b2d5874 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:45.914235 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:45.926432 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e782d8f3-cd9f-4a38-ac93-904b3f93ac6f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1334 microversion: 2.20 time: 0.321313 Oct 12 02:29:45.927160 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 14/25] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:45 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1334 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:29:45.931665 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cb7bcbb2-2869-4f8f-8248-d2669b2d5874 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1222 microversion: 2.1 time: 0.455969 Oct 12 02:29:45.941086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 12/26] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:45 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1222 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:46.489283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:46.493141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:46.518116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:46.518610 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:46.519046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:46.541141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:46.541760 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:46.544456 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:46.903188 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:46.915751 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5e33c432-7412-4e1b-a7f8-8de096799eb5 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.431802 Oct 12 02:29:46.916762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 15/27] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:46 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:46.962679 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:46.963296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:46.967953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:46.968820 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:46.980164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:46.981771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:46.982945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:46.985547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:46.986341 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:46.990460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:47.011297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:47.011786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:47.012241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:47.013018 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:47.013482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:47.013914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:47.132390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:47.141738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e545a327-3aef-4095-b40f-f5bb24453b01 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.178211 Oct 12 02:29:47.156725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 16/28] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:46 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:47.338154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:47.352017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:47.353866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:47.355009 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a8c8f137-37ad-4487-9d8d-e9711fcb643f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1334 microversion: 2.20 time: 0.399438 Oct 12 02:29:47.372446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 13/29] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:46 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1334 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:47.941510 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:47.943177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:47.956248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-378175626", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "21fdf3f0-6b90-400a-b2fc-5e540c8c18e8"}], "security_groups": [{"name": "tempest-securitygroup--1123924197"}], "key_name": "tempest-keypair-780906542", "user_data": "}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:47.960313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:47.961833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:47.962279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:47.962977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:47.963428 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:47.963845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:48.136966 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:48.147827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3201a0d0-4637-4eba-b5a3-754b888a354a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.211359 Oct 12 02:29:48.165595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 17/30] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:47 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:48.190657 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:48.191536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:48.206266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:48.208594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:48.210201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:48.212053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:48.213429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:48.215398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:48.346223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:48.346891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] validate_networks() for [('21fdf3f0-6b90-400a-b2fc-5e540c8c18e8', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:29:48.358894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fe6ee726-665d-447a-8a7a-8b173cd973b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.189139 Oct 12 02:29:48.369876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 18/31] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:48 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:48.402950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:48.403615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:48.414250 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:48.415466 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:48.416108 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:48.417698 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:48.418347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:48.418901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:48.572212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:48.586989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:48.588545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:48.589741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-217ac665-e75c-4986-b13e-b1b565f945e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1334 microversion: 2.20 time: 0.192476 Oct 12 02:29:48.600951 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 19/32] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:48 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1334 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:48.956861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-a6d259b8-cc1e-477a-a8f2-0217c91d2699 req-efe4a356-99bc-460b-924e-648d91bbb623 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a23966c6-4a75-42ef-8980-9d017342ee61", "tag": "4926fd07-e534-4077-a727-12393bfe3a35"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:29:48.973737 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a6d259b8-cc1e-477a-a8f2-0217c91d2699 req-efe4a356-99bc-460b-924e-648d91bbb623 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:48.974221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a6d259b8-cc1e-477a-a8f2-0217c91d2699 req-efe4a356-99bc-460b-924e-648d91bbb623 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:48.974653 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a6d259b8-cc1e-477a-a8f2-0217c91d2699 req-efe4a356-99bc-460b-924e-648d91bbb623 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.016996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-a6d259b8-cc1e-477a-a8f2-0217c91d2699 req-efe4a356-99bc-460b-924e-648d91bbb623 service nova] Creating event network-changed:4926fd07-e534-4077-a727-12393bfe3a35 for instance a23966c6-4a75-42ef-8980-9d017342ee61 on hv-cinder-84914 Oct 12 02:29:49.016996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a6d259b8-cc1e-477a-a8f2-0217c91d2699 req-efe4a356-99bc-460b-924e-648d91bbb623 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118865) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 12 02:29:49.016996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a6d259b8-cc1e-477a-a8f2-0217c91d2699 req-efe4a356-99bc-460b-924e-648d91bbb623 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118865) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 12 02:29:49.016996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.016996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.021238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.022315 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.023119 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.023846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.039068 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.compute.rpcapi [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 12 02:29:49.040309 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] Releasing lock "compute-rpcapi-router" {{(pid=118865) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 12 02:29:49.058441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bf6d5b5a-9970-4fc2-8801-b1db6be07c60 None None] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.174309 Oct 12 02:29:49.059702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 20/33] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:29:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:29:49.120999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:29:49.121427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:29:49.121895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:29:49.122297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:29:49.122699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:29:49.131381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.131859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.132287 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.133043 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.133484 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.133897 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.151614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:29:49.155359 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:29:49.166640 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:29:49.168822 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:29:49.171953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:49.172710 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:49.178923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.179403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.179836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.180710 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.181184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.181600 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.219031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-147ceae0-530b-4ab9-8cc7-aaf3de26da71 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.269244 Oct 12 02:29:49.219784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 14/34] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:29:47 2022] POST /compute/v2.1/servers => generated 402 bytes in 1271 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 12 02:29:49.239267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2657fc3d-8d8a-49ee-8f29-0e05e4690d1b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:49.240242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2657fc3d-8d8a-49ee-8f29-0e05e4690d1b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:49.303138 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2657fc3d-8d8a-49ee-8f29-0e05e4690d1b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:49.312522 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2657fc3d-8d8a-49ee-8f29-0e05e4690d1b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1240 microversion: 2.1 time: 0.084747 Oct 12 02:29:49.313528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 15/35] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:49 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1240 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:49.325727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:49.333895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e5be959a-7359-4539-b224-0bf1f30bb008 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.166201 Oct 12 02:29:49.334453 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 21/36] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:49 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:49.385174 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:49.385781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:49.391716 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.392194 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.392627 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.393406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.393853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.394270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.455036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:49.463645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3a823299-8e1a-45ac-85ff-ac6a7f2ef569 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.082203 Oct 12 02:29:49.464283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 16/37] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:49 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:49.633398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:49.634146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:49.653169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.653735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.654347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.655107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:49.655672 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:49.656106 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:49.767474 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:49.777028 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:49.778484 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:49.779592 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f5ee7446-eff6-447a-b020-4ccce9687181 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.156914 Oct 12 02:29:49.780153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 22/38] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:49 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:50.337271 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:50.338198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:50.344438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.345447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.345838 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:50.346799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.347449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.348083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:50.358156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:50.358818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:50.370322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.371076 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.371513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:50.372263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.372731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.373218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:50.453550 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:50.471834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8d77cc65-e05d-4b2d-bb89-a50251b8f1e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.138277 Oct 12 02:29:50.472746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 17/39] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:50 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:50.492011 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:50.492628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:50.500455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.500977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.501713 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:50.502651 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.503176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.503797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:50.563939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:50.573913 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:50.583906 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4d536880-0dc1-4a93-ad95-eea78727a325 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.101228 Oct 12 02:29:50.584967 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 18/40] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:50 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:50.588725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2536113e-1b06-4d5d-b324-d53415d74a6b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.233616 Oct 12 02:29:50.590132 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 23/41] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:50 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:50.810873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:50.811657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:50.835522 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.836028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.836465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:50.837205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:50.837662 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:50.838085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:51.041769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:51.060574 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:51.070609 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:51.071762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bde6b20c-d8d1-4be2-a655-4bb0267e6f92 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.267354 Oct 12 02:29:51.072624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 19/42] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:50 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:29:51.498365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:51.499049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:51.506076 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:51.506659 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:51.507159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:51.508071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:51.508587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:51.509110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:51.586833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:51.599842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6297015b-7e77-4fa1-898c-50e9b85b0cbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.107044 Oct 12 02:29:51.600549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 24/43] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:51 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:51.613867 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:51.614617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:51.615738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:51.618008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:51.625037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:51.625463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:51.625953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:51.626743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:51.627282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:51.627766 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:51.628123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:51.628628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:51.629101 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:51.633333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:51.643297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:51.643876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:51.735034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:51.749320 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-655c23e2-61b4-4d27-89a2-133e7cc1ece5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.145502 Oct 12 02:29:51.750465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 20/44] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:51 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:51.906527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:51.915485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-03cdd66f-1703-4651-aaa8-17e47c550cfa tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.306403 Oct 12 02:29:51.916068 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 25/45] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:51 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:52.107749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:52.108375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:52.116909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.117407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.121256 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:52.122024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.123075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.123533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:52.206832 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:52.219747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:52.222276 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:52.224063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d3ada365-e756-481b-9fa5-98fa7aa61192 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.120378 Oct 12 02:29:52.225955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 21/46] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:52 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:52.625634 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:52.626998 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:52.633989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.635523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.636577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:52.637597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.638717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.639338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:52.732913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:52.745099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-686d835d-86b8-4b75-8389-b1d229856552 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.123657 Oct 12 02:29:52.745843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 26/47] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:52 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:52.778973 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:52.779821 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:52.789302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.790017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.790787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:52.791824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.792268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.792735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:52.868596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:52.876391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b85b39fd-0dac-49ca-8364-340e40d32380 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.104764 Oct 12 02:29:52.876986 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 22/48] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:52 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:52.941616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:52.942925 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:52.945992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:52.959252 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.960071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.960794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:52.961718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:52.962442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:52.963014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:53.148447 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:53.158238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-92339228-76be-4c64-8d14-7d52878e9523 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.221031 Oct 12 02:29:53.159019 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 27/49] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:52 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:53.250855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:53.251746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:53.258561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:53.259048 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:53.259472 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:53.260500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:53.261052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:53.261813 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:53.349620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:53.362763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:53.365093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:53.366594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b4915f9c-ff54-460f-9297-4091813c9a0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.119820 Oct 12 02:29:53.368310 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 23/50] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:53 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:53.779061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:53.779607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:53.788539 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:53.790244 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:53.791090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:53.792337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:53.810176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:53.821291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:53.902520 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:53.903305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:53.911989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:53.914727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:53.915455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:53.917524 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:53.918951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:53.921182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:53.939081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:53.943967 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:53.954044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ca288b64-06d9-4128-94ec-ea8dcffad494 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.178395 Oct 12 02:29:53.956435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 28/51] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:53 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:54.005092 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:54.014681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-95917a04-21c4-479d-8ca6-ef860bc82665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.116607 Oct 12 02:29:54.015719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 24/52] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:53 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:54.182968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:54.189487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:54.200182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:54.211906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:54.211906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:54.211906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:54.211906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:54.212184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:54.407329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:54.408027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:54.416515 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:54.418955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:54.420182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:54.421248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:54.422387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:54.422986 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:54.451371 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=28 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:54.516489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:54.528585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:54.530681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:54.532354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:54.534268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-398b3dd9-f8c0-41b9-8b8a-d9b8865bfd55 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.139270 Oct 12 02:29:54.535259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 25/53] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:54 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:54.536822 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d3347325-c125-491a-8084-94db527bb48a tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.358054 Oct 12 02:29:54.552357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 29/54] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:54 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:54.981760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:54.983114 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:54.996869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:54.997417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:54.997961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:54.998802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:54.999428 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:54.999950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:55.038593 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:55.039780 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:55.048658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:55.049415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:55.049999 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:55.050697 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:55.051652 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:55.052936 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:55.096008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:55.112740 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:55.132003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-23363631-a2fd-4674-aae4-b52588bf39e9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.144851 Oct 12 02:29:55.133238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 26/55] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:54 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:55.263450 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:55.282767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fc52c871-db42-4bca-922c-fe81a75cca6c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.248036 Oct 12 02:29:55.283345 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 30/56] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:55 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:55.561931 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:55.562737 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:55.566079 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:55.580822 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:55.586943 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:55.587736 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:55.591126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:55.596342 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:55.598190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:55.598667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:55.600411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:55.602663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:55.605181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:55.605993 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:55.606638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:55.607687 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:55.608257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:55.687112 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:55.699093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:55.700748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:55.713395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3998839b-151e-4407-9ffe-8b6886f643cb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.155139 Oct 12 02:29:55.715824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 27/57] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:55 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:55.845623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:55.857376 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1d288aa5-31ba-4114-a65f-4bbfed422319 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.299327 Oct 12 02:29:55.858165 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 31/58] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:55 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:56.159019 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:56.159733 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:56.166627 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.167891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.168604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:56.173688 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.174492 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.175105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:56.295592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:56.305385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-785995a7-d250-4cb4-8dac-077cc772f3d0 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.149621 Oct 12 02:29:56.305385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 28/59] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:56 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:56.310509 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:56.311117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:56.328207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.331060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.331649 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:56.333393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.335149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.337275 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:56.419322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:56.437176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-05d69ea7-4f61-49fb-ad2b-489189e40731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.129190 Oct 12 02:29:56.452724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 32/60] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:56 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:56.741226 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:56.742050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:56.748115 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.748620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.749192 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:56.749976 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.750538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.751056 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:56.836490 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:56.845488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:56.846914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:56.848175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c3f55b37-3893-47ea-914f-f4a88d44af48 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.112165 Oct 12 02:29:56.849261 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 29/61] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:56 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:56.884864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:56.885476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:56.893680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.894369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.894927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:56.896205 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:56.896750 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:56.897240 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:57.049566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:57.059806 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ecb1146a-d32f-42d9-bf79-37fc7d6bb06e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.182661 Oct 12 02:29:57.060512 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 33/62] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:56 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:29:57.331863 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:57.332504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:57.338632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:57.339119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:57.339551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:57.340234 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:57.340728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:57.341200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:57.407004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:57.415060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6528887f-6b2c-493a-80f2-ed11eda6b2f7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.086994 Oct 12 02:29:57.415620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 30/63] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:57 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:57.462091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:57.462698 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:57.468574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:57.469091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:57.469585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:57.470283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:57.470735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:57.471155 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:57.531922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:57.539587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b701cd0d-f65f-4768-b6d0-d84e69b25bc6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.081319 Oct 12 02:29:57.540239 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 34/64] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:57 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:57.873902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:57.874502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:57.882020 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:57.882460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:57.883034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:57.883776 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:57.884228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:57.884644 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:57.953723 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:57.962398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:57.963827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:57.965033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ce16133b-6396-4e84-b725-8caefab40ba8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.095021 Oct 12 02:29:57.965869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 31/65] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:57 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:58.084016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:58.084767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:58.090625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.091043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.091469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:58.092169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.092611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.093115 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:58.237553 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:58.247070 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d21db8f1-d4ad-4b17-b31e-1968c0f3f439 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.167510 Oct 12 02:29:58.262100 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 35/66] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:58 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:58.438580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:58.439109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:58.446805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.447279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.447845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:58.448530 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.449040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.449458 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:58.511350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:58.518812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4fbedc26-27eb-4efb-8f04-6d8ca85c10fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1244 microversion: 2.1 time: 0.084364 Oct 12 02:29:58.519364 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 32/67] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:58 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1244 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:58.562534 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:58.563142 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:58.569562 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.570047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.570476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:58.571169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.571619 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.572042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:58.631378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:58.638913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fdad1d5d-5916-4d96-944a-96798000c951 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.080229 Oct 12 02:29:58.639477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 36/68] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:58 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:58.990371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:58.991008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:58.997274 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.997762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.998197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:58.999027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:58.999519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:58.999944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:59.061017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:59.069217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:59.070658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:59.071748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d58c0872-cc5b-4d86-9f20-695d91599b16 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1342 microversion: 2.20 time: 0.085998 Oct 12 02:29:59.072296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 33/69] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:29:58 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1342 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:29:59.271958 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:59.272597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:59.274463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=8 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:59.279876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:59.280361 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:59.280824 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:59.281515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:59.281967 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:59.282393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:59.553312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:59.554045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:59.555943 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=4 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:59.556688 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=24 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:29:59.568405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:59.569148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:59.569806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:59.570647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:59.571239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:59.571824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:59.610879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:59.622138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c56c0705-4fc4-4309-ab9b-0ae9716e629f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.354362 Oct 12 02:29:59.624574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 37/70] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:59 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:59.663845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:29:59.673656 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:29:59.683299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:59.683823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:59.684343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:59.685090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:29:59.685599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:29:59.686213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:29:59.696491 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:59.706748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-51a917f2-1e36-47ab-a018-7e16e011f0a3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1236 microversion: 2.1 time: 0.159852 Oct 12 02:29:59.708846 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 34/71] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:59 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1236 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:29:59.799147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:29:59.818213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1fc27da9-01aa-4311-be85-6b46af6a4f40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1218 microversion: 2.1 time: 0.158810 Oct 12 02:29:59.828561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 38/72] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:29:59 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1218 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:00.098266 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:00.099269 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:00.105991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.106793 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.107566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:00.108395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.109401 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.110004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:00.424847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:00.441245 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:00.444565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:00.446604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-745b3cc8-55a7-4757-9182-48767be0716d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.353426 Oct 12 02:30:00.447850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 35/73] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:00 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:30:00.650525 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:00.651124 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:00.686267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.690122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.694861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:00.698174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.699262 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.700880 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:00.735516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:00.736873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:00.753241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.755579 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.756295 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:00.757306 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.758221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.758857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:00.944406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:00.954278 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-17a2fa91-3963-4908-a9a2-eace403fded2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1236 microversion: 2.1 time: 0.224130 Oct 12 02:30:00.956485 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 36/74] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:00 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1236 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:00.968250 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:00.968818 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:00.980471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.980991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.981418 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:00.988005 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:00.990885 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:00.991405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:01.115205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:01.139267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cd2423aa-8390-45b0-9018-10fe1c285685 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1274 microversion: 2.1 time: 0.171760 Oct 12 02:30:01.167489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 37/75] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:00 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1274 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:01.302862 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:01.327804 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5bb3c14d-9592-4c90-a952-13567ec668ac tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.684522 Oct 12 02:30:01.328715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 39/76] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:00 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:01.477288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:01.478110 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:01.514170 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:01.516639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:01.520086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:01.520780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:01.523695 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:01.524533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:01.964629 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:01.979757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:01.982826 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:01.984722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:01.985399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:01.985731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f135c183-debe-4024-90e2-20ba38df1658 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.513003 Oct 12 02:30:02.001260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.001260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.001260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.001260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.002121 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.002121 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.004495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 38/77] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:01 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:30:02.081759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:02.093182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8f9920e8-36bb-40b3-92b5-06f3105cd374 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1292 microversion: 2.1 time: 0.112664 Oct 12 02:30:02.122069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-f6e8cd41-2c7f-47c4-ad5b-f3b87ce4e355 req-381fde7e-1949-411a-85c1-abe5805d6566 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48eb4bd4-2680-4fc6-85c0-dae85fb740fa", "tag": "0a31a866-f72c-4f7d-822f-7d54a4e3afec"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:02.123640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 40/78] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:01 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1292 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:02.159997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f6e8cd41-2c7f-47c4-ad5b-f3b87ce4e355 req-381fde7e-1949-411a-85c1-abe5805d6566 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.162293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f6e8cd41-2c7f-47c4-ad5b-f3b87ce4e355 req-381fde7e-1949-411a-85c1-abe5805d6566 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.164394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f6e8cd41-2c7f-47c4-ad5b-f3b87ce4e355 req-381fde7e-1949-411a-85c1-abe5805d6566 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.173426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:02.174312 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:02.189201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.189817 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.190842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.194906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.196739 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.197442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.242789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-f6e8cd41-2c7f-47c4-ad5b-f3b87ce4e355 req-381fde7e-1949-411a-85c1-abe5805d6566 service nova] Creating event network-changed:0a31a866-f72c-4f7d-822f-7d54a4e3afec for instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa on hv-cinder-84914 Oct 12 02:30:02.243325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f6e8cd41-2c7f-47c4-ad5b-f3b87ce4e355 req-381fde7e-1949-411a-85c1-abe5805d6566 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118864) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 12 02:30:02.245466 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f6e8cd41-2c7f-47c4-ad5b-f3b87ce4e355 req-381fde7e-1949-411a-85c1-abe5805d6566 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118864) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 12 02:30:02.256657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.257408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.258076 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.258896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.259356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.259805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.283990 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.compute.rpcapi [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 12 02:30:02.287052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] Releasing lock "compute-rpcapi-router" {{(pid=118864) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 12 02:30:02.317866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cbb5f721-c6bf-47ef-ba18-38adde89b515 None None] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.200728 Oct 12 02:30:02.319143 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 39/79] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:02.378264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:02.382380 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:02.394959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:02.400093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.400784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.402542 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.404234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c780427c-eaa6-4b6d-a9fa-addfe06f26ea tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1280 microversion: 2.1 time: 0.237952 Oct 12 02:30:02.405024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:02.405541 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:02.405985 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:02.408529 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 41/80] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:02 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1280 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:02.903403 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:02.914028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-18c8045a-5e8b-4602-bc6c-de87f4eca52f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.548533 Oct 12 02:30:02.914028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 40/81] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:02 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:03.016391 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:03.018624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:03.028535 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.029941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.030981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:03.031882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.033023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.033648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:03.116754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:03.117997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:03.131783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.134241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.134864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:03.136775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.137442 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.138035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:03.304412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:03.320726 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6f4362ea-0b30-4c25-8587-ee58bc081354 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1300 microversion: 2.1 time: 0.208400 Oct 12 02:30:03.323499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 41/82] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:03 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1300 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:03.438394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:03.439150 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:03.447588 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.448367 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.449120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:03.450168 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.450921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.451964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:03.586223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:03.599783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:03.601646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:03.603677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2a1b5e01-e66d-46cb-8260-025abc444378 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.591474 Oct 12 02:30:03.605192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 42/83] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:03 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:30:03.936202 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:03.937106 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:03.948616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.949564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.951629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:03.952639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:03.953865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:03.954690 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:04.068113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:04.088491 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-895f9aa9-1652-499c-bed3-4a945cd01190 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.644430 Oct 12 02:30:04.089378 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 42/84] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:03 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:04.349308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:04.349907 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:04.362819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:04.374193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:04.374726 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:04.375030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bca8a32b-378b-4b68-ba9c-ecb55a129b7c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.440527 Oct 12 02:30:04.377311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:04.382008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:04.384682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:04.388471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:04.392757 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 43/85] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:03 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:04.636462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:04.637254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:04.651799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:04.652397 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:04.652862 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:04.653811 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:04.654478 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:04.655094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:04.686388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:04.690572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:04.701205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5d003e5b-4580-4ea5-9d51-4439fe41d509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1298 microversion: 2.1 time: 0.356040 Oct 12 02:30:04.702564 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 43/86] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:04 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1298 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:05.032327 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:05.048800 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:05.051331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:05.052717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-16c782e9-2b12-405a-85b4-15b48ff8fd0b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.420892 Oct 12 02:30:05.069750 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 44/87] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:04 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:30:05.116686 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:05.117398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:05.129907 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:05.130679 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:05.135147 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:05.136063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:05.142952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:05.143754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:05.399706 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:05.400307 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:05.412182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:05.413112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:05.413946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:05.414857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:05.416828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:05.419159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:05.447364 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:05.470426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d1be5b48-49ef-4efe-8af1-05fb52bd210e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.357958 Oct 12 02:30:05.473907 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 44/88] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:05 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:05.481362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:06.076811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-48bea2b1-0eea-4ec7-963d-66e341fa4aca req-cb6e0727-c65a-45ef-bbed-6157089794b2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3d9e5223-f388-4f16-b15a-73c59e86de18", "tag": "1ebc7cae-9780-4ccb-8ae6-cdd2a9b8caee"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:06.111314 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-48bea2b1-0eea-4ec7-963d-66e341fa4aca req-cb6e0727-c65a-45ef-bbed-6157089794b2 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.119811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-48bea2b1-0eea-4ec7-963d-66e341fa4aca req-cb6e0727-c65a-45ef-bbed-6157089794b2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.121555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-48bea2b1-0eea-4ec7-963d-66e341fa4aca req-cb6e0727-c65a-45ef-bbed-6157089794b2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:06.163744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-48bea2b1-0eea-4ec7-963d-66e341fa4aca req-cb6e0727-c65a-45ef-bbed-6157089794b2 service nova] Creating event network-changed:1ebc7cae-9780-4ccb-8ae6-cdd2a9b8caee for instance 3d9e5223-f388-4f16-b15a-73c59e86de18 on hv-cinder-84914 Oct 12 02:30:06.197548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-48bea2b1-0eea-4ec7-963d-66e341fa4aca req-cb6e0727-c65a-45ef-bbed-6157089794b2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.541507 Oct 12 02:30:06.199715 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 45/89] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:06.214877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:06.218414 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:06.234574 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.237546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.240957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:06.242931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.243434 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.245027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:06.330744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:06.340275 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-480b2fdc-95c5-45ae-ae47-b50513abbaa7 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.944573 Oct 12 02:30:06.345668 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 45/90] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:05 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 951 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:06.357050 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-f795ca8c-ce84-4cb3-9a0c-a41288f92834 req-59f61942-ecba-4fc7-94a9-8ec2d8be1f5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "44170331-47f8-456c-ad56-e86eab5cac15", "tag": "083d5d6b-4001-402b-be73-2549ed579b58"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:06.385177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f795ca8c-ce84-4cb3-9a0c-a41288f92834 req-59f61942-ecba-4fc7-94a9-8ec2d8be1f5e service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.386666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f795ca8c-ce84-4cb3-9a0c-a41288f92834 req-59f61942-ecba-4fc7-94a9-8ec2d8be1f5e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.388355 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f795ca8c-ce84-4cb3-9a0c-a41288f92834 req-59f61942-ecba-4fc7-94a9-8ec2d8be1f5e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:06.417351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-f795ca8c-ce84-4cb3-9a0c-a41288f92834 req-59f61942-ecba-4fc7-94a9-8ec2d8be1f5e service nova] Creating event network-changed:083d5d6b-4001-402b-be73-2549ed579b58 for instance 44170331-47f8-456c-ad56-e86eab5cac15 on hv-cinder-84914 Oct 12 02:30:06.430120 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-f795ca8c-ce84-4cb3-9a0c-a41288f92834 req-59f61942-ecba-4fc7-94a9-8ec2d8be1f5e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083136 Oct 12 02:30:06.431093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 46/91] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:06.437418 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:06.438506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:06.446866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.446866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.446866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:06.446866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.447920 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.447920 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:06.741856 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:06.752454 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f58eecb5-2ce2-450f-aee3-baa14c0bffcf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.548557 Oct 12 02:30:06.754398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 46/92] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:06 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:06.768594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:06.769449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:06.781779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.782555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.783179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:06.784032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:06.784683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:06.786960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:07.118316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:07.126837 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:07.128369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:07.130255 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bf2c6dbb-b82b-4bee-8fed-12f21f30ecbb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.698089 Oct 12 02:30:07.131046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 47/93] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:06 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 699 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:30:07.147246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:07.157395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0fa79452-30b0-4dcb-a695-ba2609f9c590 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.401814 Oct 12 02:30:07.158216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 47/94] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:06 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:07.380590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:07.381918 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:07.390136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:07.391023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:07.391675 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:07.392627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:07.393747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:07.394611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:07.614954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:07.630418 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-83ca3e63-ac1b-454a-a8b0-c4954ee7277e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.260722 Oct 12 02:30:07.632283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 48/95] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:07 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:07.778147 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:07.779347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:07.787149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:07.787609 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:07.788167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:07.788817 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:07.789271 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:07.789746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:07.938032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:07.946880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-333d9246-26b5-4c2b-a7cb-8ed6f0c13244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.173442 Oct 12 02:30:07.947624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 48/96] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:07 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:08.156758 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:08.157598 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:08.170040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.170712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.171290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.172162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.172645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.176514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.182761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:08.183653 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:08.199199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.199885 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.202498 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.203384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.203862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.204297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.472113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:08.482677 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2d672037-2b20-4b57-bb1e-3beacf8ae1fa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.304677 Oct 12 02:30:08.484391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 49/97] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:08 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:08.555657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-894d3c84-eeaa-4616-86c3-85ecebcd9be0 req-1a5a1d95-f7f3-472f-b26a-9ae4e69281e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a23966c6-4a75-42ef-8980-9d017342ee61", "name": "network-vif-plugged", "status": "completed", "tag": "4926fd07-e534-4077-a727-12393bfe3a35"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:08.565719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-894d3c84-eeaa-4616-86c3-85ecebcd9be0 req-1a5a1d95-f7f3-472f-b26a-9ae4e69281e4 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.566225 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-894d3c84-eeaa-4616-86c3-85ecebcd9be0 req-1a5a1d95-f7f3-472f-b26a-9ae4e69281e4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.566810 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-894d3c84-eeaa-4616-86c3-85ecebcd9be0 req-1a5a1d95-f7f3-472f-b26a-9ae4e69281e4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.579132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-894d3c84-eeaa-4616-86c3-85ecebcd9be0 req-1a5a1d95-f7f3-472f-b26a-9ae4e69281e4 service nova] Creating event network-vif-plugged:4926fd07-e534-4077-a727-12393bfe3a35 for instance a23966c6-4a75-42ef-8980-9d017342ee61 on hv-cinder-84914 Oct 12 02:30:08.584943 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-894d3c84-eeaa-4616-86c3-85ecebcd9be0 req-1a5a1d95-f7f3-472f-b26a-9ae4e69281e4 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099105 Oct 12 02:30:08.585548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 50/98] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:08.591020 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-d941c4a7-352f-4cc1-ade5-a33f58339ace req-b43e327e-d1f6-47ce-bc8c-812ec4525cea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "76667a6b-65ab-4a9d-8af3-0440b0dad579", "tag": "a0229b37-24b4-4585-84cc-d3276e3660d3"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:08.601538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d941c4a7-352f-4cc1-ade5-a33f58339ace req-b43e327e-d1f6-47ce-bc8c-812ec4525cea service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.602214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d941c4a7-352f-4cc1-ade5-a33f58339ace req-b43e327e-d1f6-47ce-bc8c-812ec4525cea service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.602893 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d941c4a7-352f-4cc1-ade5-a33f58339ace req-b43e327e-d1f6-47ce-bc8c-812ec4525cea service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.617047 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-d941c4a7-352f-4cc1-ade5-a33f58339ace req-b43e327e-d1f6-47ce-bc8c-812ec4525cea service nova] Creating event network-changed:a0229b37-24b4-4585-84cc-d3276e3660d3 for instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 on hv-cinder-84914 Oct 12 02:30:08.622181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-d941c4a7-352f-4cc1-ade5-a33f58339ace req-b43e327e-d1f6-47ce-bc8c-812ec4525cea service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035128 Oct 12 02:30:08.635218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:08.635802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 51/99] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:08.645065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:08.646586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:08.647693 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7c7a02cb-34ea-4b53-b897-f3a5ade2e49c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.495932 Oct 12 02:30:08.648408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 49/100] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:08 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:30:08.670669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:08.671543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:08.679351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.680440 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.684226 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.686187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.686666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.687094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.933076 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:08.951205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1e5806b8-8b87-4453-8bdc-095076d77c3c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.284488 Oct 12 02:30:08.951959 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 52/101] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:08 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:08.974837 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:08.975537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:08.982916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.983643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.984410 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:08.985364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:08.986017 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:08.986626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:09.154994 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:09.163647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e8cae44d-b2c3-4ca6-8881-a5ac52b7a4d8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.193181 Oct 12 02:30:09.168870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 50/102] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:08 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:09.508875 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:09.510964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:09.526062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:09.527622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:09.528478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:09.529601 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:09.530587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:09.531279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:09.672089 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:09.672551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:09.680001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:09.688281 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:09.689038 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:09.689917 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:09.690493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:09.691581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:09.888337 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:09.899757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4ba4b583-c9cf-49fb-9b66-8fd4b096eb1d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.396107 Oct 12 02:30:09.901219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 53/103] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:09 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:09.919721 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:09.929645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:09.931132 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:09.932189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-82811f2e-7590-4dc4-a775-af50d4fa744a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.264168 Oct 12 02:30:09.933079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 51/104] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:09 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:30:09.976486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:09.978019 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:09.986348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:09.986846 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:09.987277 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:09.990013 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:09.991854 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:09.992450 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:10.161684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:10.169681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5ebfd9de-72c8-4e48-a9e9-9fa495b87691 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.196780 Oct 12 02:30:10.170786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 54/105] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:09 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:10.193264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:10.194000 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:10.200635 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:10.201268 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:10.201892 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:10.202753 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:10.203206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:10.203627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:10.356437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:10.365878 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-96e2dea2-e346-4656-b974-7d3b4df03f0e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.178119 Oct 12 02:30:10.366771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 52/106] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:10 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:10.927260 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:10.928198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:10.935607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:10.936393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:10.936930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:10.937924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:10.938474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:10.939013 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:10.957915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:10.959151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:10.969895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:10.970380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:10.971146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:10.972017 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:10.973724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:10.974322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:11.157808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:11.167439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c4dc6f5c-8476-4d2c-ab3c-da7916d55cf7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.246777 Oct 12 02:30:11.168638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 55/107] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:10 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:11.199166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:11.200271 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:11.208839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:11.209547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:11.210200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:11.211533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:11.213098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:11.213766 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:11.282502 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:11.292247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:11.297768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:11.299257 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-02923bc2-865c-4696-9b09-8b84e7558e7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.345645 Oct 12 02:30:11.305040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 53/108] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:10 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:30:11.391783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:11.393513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:11.418273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:11.418976 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:11.420185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:11.421552 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:11.422490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:11.431336 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:11.825766 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:11.834177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f0f9cc16-1ee5-4d3f-8cba-88f800d2543f tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.641475 Oct 12 02:30:11.836839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 56/109] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:11 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:12.197176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:12.205542 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:12.223684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:12.224874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:12.225480 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:12.226795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:12.227245 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:12.227808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:12.233609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:12.249200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-69e38f7f-ee48-4310-a6b6-91a37e6ee79b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.857931 Oct 12 02:30:12.250561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 54/110] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:11 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 865 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:12.283878 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:12.334027 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:12.335271 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:12.351328 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:12.352216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:12.352655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:12.353386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:12.354582 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:12.359820 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:12.513656 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:12.526294 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9d463433-ccd8-497a-96cc-2ba78b3e52fc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.338515 Oct 12 02:30:12.540724 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 57/111] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:12 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:12.785467 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:12.796260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:12.797781 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:12.799235 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b867c49e-c127-45f8-b0dc-9b3df58f0cbf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.476892 Oct 12 02:30:12.799982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 55/112] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:12 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:30:12.858547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:12.859193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:12.866971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:12.867598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:12.868189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:12.869343 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:12.869940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:12.870416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.278646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:13.279581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:13.301820 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:13.306551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:13.307660 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.310984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:13.311591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:13.312149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.351973 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:13.354010 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-dba0ddab-f44f-492c-ba64-57b38bc40154 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.499117 Oct 12 02:30:13.357069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 58/113] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:12 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:13.550180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:13.551397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:13.563334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:13.564020 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:13.565252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.567354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:13.568857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:13.569452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.645422 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:13.654374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ace1ca37-0b25-454b-934f-d7dce9d33db5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.382103 Oct 12 02:30:13.657660 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 56/114] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:13 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:13.691715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-72e722a2-b2c6-43a5-83bc-78eb43576856 req-b415d5d5-a0a3-4e7b-ab96-ab338ee08954 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d9e5223-f388-4f16-b15a-73c59e86de18", "name": "network-vif-plugged", "status": "completed", "tag": "1ebc7cae-9780-4ccb-8ae6-cdd2a9b8caee"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:13.716682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-72e722a2-b2c6-43a5-83bc-78eb43576856 req-b415d5d5-a0a3-4e7b-ab96-ab338ee08954 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:13.718140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-72e722a2-b2c6-43a5-83bc-78eb43576856 req-b415d5d5-a0a3-4e7b-ab96-ab338ee08954 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:13.718887 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-72e722a2-b2c6-43a5-83bc-78eb43576856 req-b415d5d5-a0a3-4e7b-ab96-ab338ee08954 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.762202 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-72e722a2-b2c6-43a5-83bc-78eb43576856 req-b415d5d5-a0a3-4e7b-ab96-ab338ee08954 service nova] Creating event network-vif-plugged:1ebc7cae-9780-4ccb-8ae6-cdd2a9b8caee for instance 3d9e5223-f388-4f16-b15a-73c59e86de18 on hv-cinder-84914 Oct 12 02:30:13.768832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-72e722a2-b2c6-43a5-83bc-78eb43576856 req-b415d5d5-a0a3-4e7b-ab96-ab338ee08954 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082853 Oct 12 02:30:13.769903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 57/115] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:13.826710 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:13.827485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:13.839678 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:13.841692 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:13.842333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.843869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:13.843869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:13.846824 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:13.856510 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:13.871811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-90bbbf2f-9ba1-478f-b55e-3424bb3bc4d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.324733 Oct 12 02:30:13.884861 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 59/116] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:13 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:14.035988 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:14.047137 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:14.048603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:14.049797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2719ce12-3e42-401d-b63d-eb2865f954bb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.229115 Oct 12 02:30:14.051534 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 58/117] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:13 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:30:14.378776 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:14.380624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:14.386895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:14.387580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:14.388148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:14.389008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:14.389594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:14.390150 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:14.676957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:14.679751 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:14.680358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:14.685026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-dd2a6d59-3f9c-4619-a50e-b195cbb7c750 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.310287 Oct 12 02:30:14.685975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 60/118] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:14 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:14.686984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:14.687732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:14.688374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:14.689227 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:14.689818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:14.690378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:14.880602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:14.898534 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:14.899970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:14.902479 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4ff2a5af-1a7e-42c7-9e1b-f262e06b8e95 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.226484 Oct 12 02:30:14.903562 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 59/119] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:14 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:14.910021 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:14.910751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:14.911373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:14.912895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:14.913500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:14.914066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:15.086243 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:15.086906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:15.109063 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:15.110201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:15.110860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:15.114837 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:15.116826 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:15.119930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:15.148097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:15.161741 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7724fb9d-a541-4c8d-aaee-6adb6c4e1d9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.263209 Oct 12 02:30:15.163122 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 61/120] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:14 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:15.418857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:15.428481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:15.431427 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:15.432954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-71fb56ec-24e8-4557-a64f-df8250470679 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.360643 Oct 12 02:30:15.433788 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 60/121] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:15 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:30:15.708729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:15.709360 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:15.715407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:15.715896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:15.716329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:15.717045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:15.717708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:15.718133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:15.740881 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:15.793852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-8faa35dd-2246-4bee-9626-b77e4e3e8aaa req-3201e377-63c9-454a-9cca-140d65219878 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48eb4bd4-2680-4fc6-85c0-dae85fb740fa", "name": "network-vif-plugged", "status": "completed", "tag": "0a31a866-f72c-4f7d-822f-7d54a4e3afec"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:15.813971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8faa35dd-2246-4bee-9626-b77e4e3e8aaa req-3201e377-63c9-454a-9cca-140d65219878 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:15.814625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8faa35dd-2246-4bee-9626-b77e4e3e8aaa req-3201e377-63c9-454a-9cca-140d65219878 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:15.816942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8faa35dd-2246-4bee-9626-b77e4e3e8aaa req-3201e377-63c9-454a-9cca-140d65219878 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:15.844460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-8faa35dd-2246-4bee-9626-b77e4e3e8aaa req-3201e377-63c9-454a-9cca-140d65219878 service nova] Creating event network-vif-plugged:0a31a866-f72c-4f7d-822f-7d54a4e3afec for instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa on hv-cinder-84914 Oct 12 02:30:15.860133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-8faa35dd-2246-4bee-9626-b77e4e3e8aaa req-3201e377-63c9-454a-9cca-140d65219878 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070412 Oct 12 02:30:15.867465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 61/122] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:15.928316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:15.929451 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:15.936298 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:15.937041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:15.937709 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:15.938654 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:15.939316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:15.939942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:16.130176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:16.140971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-283b0c96-754b-4222-95c4-ee6a3736f8ea tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1314 microversion: 2.1 time: 0.436091 Oct 12 02:30:16.143067 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 62/123] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:15 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1314 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:16.153903 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1498077076", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "networks": [{"uuid": "cc86f2f4-a79e-41f7-816d-bf9acbcc5b5c"}]}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:16.217889 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:16.226752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-cd662229-3395-4f47-a099-08465da21c49 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.303783 Oct 12 02:30:16.227470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 62/124] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:15 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:16.249755 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:16.250849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:16.251381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] validate_networks() for [('cc86f2f4-a79e-41f7-816d-bf9acbcc5b5c', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:30:16.259802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:16.260284 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:16.261146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:16.262574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:16.263190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:16.263616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:16.413329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:16.421172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6a40d80b-0ccb-4495-9e0f-344aaff09a4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.192250 Oct 12 02:30:16.421722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 63/125] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:16 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:16.460794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:16.461546 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:16.468408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:16.468924 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:16.469354 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:16.470046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:16.470491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:16.470908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:16.475638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:16.609046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:16.621271 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:16.622719 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:16.623836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-936e1cf0-1306-4145-8427-d99839e28c33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.168043 Oct 12 02:30:16.624403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 64/126] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:16 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:30:17.143100 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:30:17.143585 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:30:17.144179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:30:17.144417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:30:17.144849 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:30:17.158206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.158963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.160271 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.161493 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.162196 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.162837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.170588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:17.171211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:17.193911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.193911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.193911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.193911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.194245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.194245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.228078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Getting quotas for project 177af31ad03c40e6b2a4a72c16451b5d. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:30:17.247213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Getting quotas for user a2ce151a58444f84b06a6e10cdebf1c3 and project 177af31ad03c40e6b2a4a72c16451b5d. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:30:17.286587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:30:17.298615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:30:17.337238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:17.345865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c7b8c461-b8f2-4dba-a2fe-0bf44c4e86e8 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1502 microversion: 2.1 time: 0.179693 Oct 12 02:30:17.346914 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 65/127] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:17 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1502 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:17.352493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:17.353564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:17.359914 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.360628 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.361357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.362257 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.363129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.363675 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.402293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-31e07876-77db-4fdc-a9f2-2c98965fdad8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.257037 Oct 12 02:30:17.405238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 63/128] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:30:16 2022] POST /compute/v2.1/servers => generated 380 bytes in 1261 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:30:17.442619 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d2dbc640-5b7c-4b1a-9c93-720264d29ed3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:17.443427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d2dbc640-5b7c-4b1a-9c93-720264d29ed3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:17.591991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d2dbc640-5b7c-4b1a-9c93-720264d29ed3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:17.600681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d2dbc640-5b7c-4b1a-9c93-720264d29ed3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1214 microversion: 2.1 time: 0.183488 Oct 12 02:30:17.601351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 64/129] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:17 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1214 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:17.634663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:17.635344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:17.665183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.665677 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.666107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.666811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.667264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.671759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.688489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:17.702361 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2580e75a-7255-4e12-aa4b-10983f6360f4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.349921 Oct 12 02:30:17.707584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 66/130] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:17 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:17.727656 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:17.729894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:17.755763 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.762196 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.762840 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:17.765023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:17.765638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:17.770237 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:18.078365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:18.089986 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:18.091891 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:18.093261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a1a4cf2f-ca9f-4a37-a0d2-7f986a273f1d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1408 microversion: 2.20 time: 0.384478 Oct 12 02:30:18.113166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 67/131] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:17 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1408 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:30:18.126386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-5991fe67-6551-4fcc-84e7-54b083b9ed0a req-ab0c14c9-22ca-4de0-9806-4c3556405085 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "44170331-47f8-456c-ad56-e86eab5cac15", "name": "network-vif-plugged", "status": "completed", "tag": "083d5d6b-4001-402b-be73-2549ed579b58"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:18.134775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:18.142659 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0e567060-e200-4ab2-86b9-99fa6cf16abc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.530857 Oct 12 02:30:18.143238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 65/132] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:17 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:18.153520 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-5991fe67-6551-4fcc-84e7-54b083b9ed0a req-ab0c14c9-22ca-4de0-9806-4c3556405085 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:18.153520 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-5991fe67-6551-4fcc-84e7-54b083b9ed0a req-ab0c14c9-22ca-4de0-9806-4c3556405085 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:18.153520 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-5991fe67-6551-4fcc-84e7-54b083b9ed0a req-ab0c14c9-22ca-4de0-9806-4c3556405085 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:18.161151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-5991fe67-6551-4fcc-84e7-54b083b9ed0a req-ab0c14c9-22ca-4de0-9806-4c3556405085 service nova] Creating event network-vif-plugged:083d5d6b-4001-402b-be73-2549ed579b58 for instance 44170331-47f8-456c-ad56-e86eab5cac15 on hv-cinder-84914 Oct 12 02:30:18.171184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-5991fe67-6551-4fcc-84e7-54b083b9ed0a req-ab0c14c9-22ca-4de0-9806-4c3556405085 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056139 Oct 12 02:30:18.171879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 68/133] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:18.626612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:18.627213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:18.633890 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:18.634575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:18.634995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:18.635678 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:18.636282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:18.636883 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:18.700028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:18.712141 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c3176b61-ca11-412b-9ac1-b6708b023444 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1218 microversion: 2.1 time: 0.089143 Oct 12 02:30:18.713347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 66/134] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:18 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1218 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:18.738219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:18.739158 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:18.746426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:18.747156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:18.747785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:18.748638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:18.750656 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:18.751326 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:18.963152 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:18.978992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e443c9c9-430a-4961-8929-f69f468d2d52 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.245536 Oct 12 02:30:18.979711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 69/135] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:18 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:19.126300 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "901ddbf2-dfbb-417b-8fab-bb5f240a17f8"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:19.128766 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:19.130080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:19.131072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:19.139117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.140108 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.140541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.141274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.141725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.142139 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.144425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.144952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.146346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.147738 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.148200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.148643 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.426136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:19.435155 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:19.436602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:19.443577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-186fcf11-c8bb-4f52-8607-e2604c7831c6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1599 microversion: 2.20 time: 0.318529 Oct 12 02:30:19.464901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 70/136] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:30:19 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1599 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:30:19.471308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:19.472799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:19.480146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.480922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.481574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.482469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.483103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.483728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.729283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:19.740977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2a6429ae-68e2-4a3d-a304-e9bf91cb19b5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.273118 Oct 12 02:30:19.742302 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 71/137] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:19 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:19.747954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:19.748613 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:19.771394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.771903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.772357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.776905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:19.779198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:19.780623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:19.981539 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-61cb6907-3156-433f-8fa6-20439c057998 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.237848 Oct 12 02:30:19.990335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 72/138] 10.222.0.23 () {60 vars in 1316 bytes} [Wed Oct 12 02:30:19 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-interface => generated 285 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:30:19.990335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:20.002565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:20.016149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:20.027196 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:20.027735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:20.028486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:20.029060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:20.029808 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:20.279412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d596d344-e05a-4840-aa1d-f268dd4a159b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "POST /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.157585 Oct 12 02:30:20.281078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 67/139] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:30:19 2022] POST /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61/os-volume_attachments => generated 194 bytes in 1161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:20.292816 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:20.294694 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:20.309164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:20.312481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:20.313194 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:20.327623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:20.330129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:20.330760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:20.332464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:20.396393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d726536d-4356-4c62-b0a4-ce935204da7b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1266 microversion: 2.1 time: 0.405890 Oct 12 02:30:20.400447 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 73/140] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:19 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1266 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:20.436987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-680cbd41-ac45-4726-bb74-8c031348e93e req-60458538-ef14-469a-b0ab-a291a71a80b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76667a6b-65ab-4a9d-8af3-0440b0dad579", "name": "network-vif-plugged", "status": "completed", "tag": "a0229b37-24b4-4585-84cc-d3276e3660d3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:20.503884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-680cbd41-ac45-4726-bb74-8c031348e93e req-60458538-ef14-469a-b0ab-a291a71a80b7 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:20.504642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-680cbd41-ac45-4726-bb74-8c031348e93e req-60458538-ef14-469a-b0ab-a291a71a80b7 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:20.505790 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-680cbd41-ac45-4726-bb74-8c031348e93e req-60458538-ef14-469a-b0ab-a291a71a80b7 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:20.554181 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-680cbd41-ac45-4726-bb74-8c031348e93e req-60458538-ef14-469a-b0ab-a291a71a80b7 service nova] Creating event network-vif-plugged:a0229b37-24b4-4585-84cc-d3276e3660d3 for instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 on hv-cinder-84914 Oct 12 02:30:20.591096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-680cbd41-ac45-4726-bb74-8c031348e93e req-60458538-ef14-469a-b0ab-a291a71a80b7 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.188880 Oct 12 02:30:20.603397 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 74/141] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:20.764399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:20.765373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:20.774114 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:20.775528 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:20.776310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:20.777493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:20.778256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:20.779042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:21.009874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:21.022934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-780f0d5b-8d01-4399-b0fa-6935aa6f0e56 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1322 microversion: 2.1 time: 0.263102 Oct 12 02:30:21.024500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 75/142] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:20 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1322 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:21.177065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:21.185166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-70ee19fb-1b1b-427c-aedd-9ca5c049dd15 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.902289 Oct 12 02:30:21.204253 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 68/143] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:20 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 922 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:21.423420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:21.424385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:21.449126 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:21.450712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:21.454294 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:21.455324 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:21.457251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:21.457884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:21.833065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:21.851982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e41cb40c-0e39-48d3-9996-6b709b9cac39 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1284 microversion: 2.1 time: 0.431117 Oct 12 02:30:21.853851 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 76/144] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:21 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1284 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:22.051727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:22.053372 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:22.062505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:22.064761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.065280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.066579 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.067293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.067749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.068176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.212204 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:22.212848 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:22.218630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:22.221164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.222820 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.223416 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.224477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.225432 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.226554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.351713 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:22.364825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b645f539-c0c2-4c05-928c-22dfae8fe767 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1516 microversion: 2.1 time: 0.317936 Oct 12 02:30:22.365429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 69/145] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:22 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1516 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:22.389956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:22.390649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:22.394746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:22.400263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-357c9dea-9f5d-4799-bb3e-fea23598de75 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.191463 Oct 12 02:30:22.401142 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 77/146] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:22 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:22.405313 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.405975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.406562 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.407448 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.407899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.408315 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.434890 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lazy-loading 'flavor' on Instance uuid 44170331-47f8-456c-ad56-e86eab5cac15 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:30:22.599210 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c2e458c3-5836-48f7-b92f-dbe8da6c5c67 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15/action" status: 200 len: 32 microversion: 2.1 time: 0.213248 Oct 12 02:30:22.600523 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 70/147] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:30:22 2022] POST /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15/action => generated 32 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 12 02:30:22.612776 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:22.614039 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:22.622815 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.624445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.631716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.633185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.633980 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.634566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.655153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-4a6991e5-04e3-415c-bcdb-57f71987239d req-bdfa6721-3b93-47d2-8932-b4424530bb95 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48eb4bd4-2680-4fc6-85c0-dae85fb740fa", "tag": "0a31a866-f72c-4f7d-822f-7d54a4e3afec"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:22.680567 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-4a6991e5-04e3-415c-bcdb-57f71987239d req-bdfa6721-3b93-47d2-8932-b4424530bb95 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.687652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-4a6991e5-04e3-415c-bcdb-57f71987239d req-bdfa6721-3b93-47d2-8932-b4424530bb95 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.689339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-4a6991e5-04e3-415c-bcdb-57f71987239d req-bdfa6721-3b93-47d2-8932-b4424530bb95 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.726267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-4a6991e5-04e3-415c-bcdb-57f71987239d req-bdfa6721-3b93-47d2-8932-b4424530bb95 service nova] Creating event network-changed:0a31a866-f72c-4f7d-822f-7d54a4e3afec for instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa on hv-cinder-84914 Oct 12 02:30:22.732853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-4a6991e5-04e3-415c-bcdb-57f71987239d req-bdfa6721-3b93-47d2-8932-b4424530bb95 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085197 Oct 12 02:30:22.733852 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 71/148] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:22.881344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:22.882334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:22.901150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:22.912359 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6cd73380-f90a-49da-80f5-2344189ae592 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.302842 Oct 12 02:30:22.913485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 78/149] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:22 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:22.914880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.915981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.916418 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:22.918899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:22.920974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:22.921652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:23.124303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:23.134809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0ad4e53f-fccc-490d-a05f-f81276752e45 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1272 microversion: 2.1 time: 0.258201 Oct 12 02:30:23.138273 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 72/150] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:22 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1272 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:23.424514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:23.425421 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:23.438545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:23.439162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:23.441639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:23.443311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:23.443909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:23.446402 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:23.714113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:23.721967 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1fa2d869-53f8-421b-a735-474e59e33a4c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1366 microversion: 2.1 time: 0.302001 Oct 12 02:30:23.726900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 79/151] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:23 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1366 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:23.936404 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:23.937311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:23.952475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:23.953139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:23.954760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:23.955975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:23.957368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:23.958228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:24.164768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:24.165510 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:24.179311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:24.181504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:24.183936 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:24.190846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:24.192456 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:24.193044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:24.271877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:24.283751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-15afe347-43fe-418e-8f39-c05d5ac5b8a4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.352045 Oct 12 02:30:24.298493 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 73/152] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:23 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:24.614863 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:24.625836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-449d7eb1-4226-4b58-a595-975b7dc89b0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.468533 Oct 12 02:30:24.646203 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 80/153] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:24 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:24.750264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:24.751571 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:24.762341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:24.763239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:24.764088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:24.766211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:24.767019 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:24.767809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:25.062302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:25.070423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-def84c44-1cb4-47b6-a99e-5afbbd74067d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1556 microversion: 2.1 time: 0.327416 Oct 12 02:30:25.074405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 74/154] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:24 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1556 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:25.094063 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:25.094618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:25.107173 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:25.107665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:25.108092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:25.108812 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:25.109419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:25.109833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:25.238405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-40c1a65c-35f5-42dc-83d4-7fb9f7c5b362 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.148153 Oct 12 02:30:25.238984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 81/155] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:30:25 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-interface => generated 284 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:25.311361 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:25.312037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:25.326710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:25.327331 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:25.328080 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:25.329365 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:25.330036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:25.330804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:25.650610 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:25.651372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:25.664910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:25.665405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:25.667768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:25.668523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:25.669006 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:25.669434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:25.834534 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:25.861837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-aa13f962-8845-48e8-941c-926116d2f6a5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.550786 Oct 12 02:30:25.869030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 75/156] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:25 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:26.417108 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:26.419501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:26.420990 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:26.421419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:26.429243 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:26.429957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:26.430596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:26.431426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:26.431777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3ffe3251-b939-4aab-981c-402b873f6aa5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.786209 Oct 12 02:30:26.432189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:26.432686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 82/157] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:25 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:26.445936 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:26.494202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:30:26.559722 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=5,nova_cell1:INSERT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:26.609648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0ead5fa9-3288-4147-95ae-3c576b29b421 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "DELETE /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 204 len: 0 microversion: 2.1 time: 0.193333 Oct 12 02:30:26.610910 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 76/158] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:30:26 2022] DELETE /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 12 02:30:26.621570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:26.622583 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:26.635860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:26.636987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:26.637668 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:26.638794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:26.639701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:26.640467 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:26.649326 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-58cd3433-f69f-4aa7-9fe3-b94162128500 req-73e6180b-4473-461f-b6dd-e0036024d9ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7db1184c-8d98-4d39-b77c-fb92bb3fc950", "tag": "b820bded-6400-480e-810e-7da24b3b68f7"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:26.671604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-58cd3433-f69f-4aa7-9fe3-b94162128500 req-73e6180b-4473-461f-b6dd-e0036024d9ff service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:26.672384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-58cd3433-f69f-4aa7-9fe3-b94162128500 req-73e6180b-4473-461f-b6dd-e0036024d9ff service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:26.673924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-58cd3433-f69f-4aa7-9fe3-b94162128500 req-73e6180b-4473-461f-b6dd-e0036024d9ff service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:26.701228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-58cd3433-f69f-4aa7-9fe3-b94162128500 req-73e6180b-4473-461f-b6dd-e0036024d9ff service nova] Creating event network-changed:b820bded-6400-480e-810e-7da24b3b68f7 for instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 on hv-cinder-84914 Oct 12 02:30:26.711952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-58cd3433-f69f-4aa7-9fe3-b94162128500 req-73e6180b-4473-461f-b6dd-e0036024d9ff service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067221 Oct 12 02:30:26.713733 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 77/159] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:26.898837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:26.899538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:26.905922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:26.906687 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:26.907375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:26.908438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:26.909690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:26.910373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:26.981312 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:26.995713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-389cfd27-2203-42f5-b27f-71e1b202904c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1554 microversion: 2.1 time: 0.380002 Oct 12 02:30:27.012777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 83/160] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:26 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1554 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:27.095722 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:27.107512 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-134c8ec3-8f94-4336-8b67-b74ab28b771f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.213016 Oct 12 02:30:27.122254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 78/161] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:26 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:27.261069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1440530592", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "networks": [{"uuid": "7c8feeb4-fea7-46ca-9f80-d14f4cc37df8"}]}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:27.342508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] validate_networks() for [('7c8feeb4-fea7-46ca-9f80-d14f4cc37df8', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:30:27.459059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:27.460747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:27.467416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:27.471916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:27.472765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:27.473419 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:27.474380 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:27.475024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:27.475659 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:27.597391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=8 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:27.674271 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:27.690195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-486d6be3-8aed-4743-aaaa-5b38ec4d33a8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.235507 Oct 12 02:30:27.704338 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 79/162] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:27 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:27.782249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-89f3495e-19f9-4df1-bb00-6b6ca7c76447 req-277b5bb3-5ac1-44c4-9bda-e18628a5776f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "76667a6b-65ab-4a9d-8af3-0440b0dad579", "tag": "a0229b37-24b4-4585-84cc-d3276e3660d3"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:27.798129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-89f3495e-19f9-4df1-bb00-6b6ca7c76447 req-277b5bb3-5ac1-44c4-9bda-e18628a5776f service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:27.798832 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-89f3495e-19f9-4df1-bb00-6b6ca7c76447 req-277b5bb3-5ac1-44c4-9bda-e18628a5776f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:27.799462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-89f3495e-19f9-4df1-bb00-6b6ca7c76447 req-277b5bb3-5ac1-44c4-9bda-e18628a5776f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:27.818844 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-89f3495e-19f9-4df1-bb00-6b6ca7c76447 req-277b5bb3-5ac1-44c4-9bda-e18628a5776f service nova] Creating event network-changed:a0229b37-24b4-4585-84cc-d3276e3660d3 for instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 on hv-cinder-84914 Oct 12 02:30:27.824372 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-89f3495e-19f9-4df1-bb00-6b6ca7c76447 req-277b5bb3-5ac1-44c4-9bda-e18628a5776f service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046049 Oct 12 02:30:27.825570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 80/163] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:28.027508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:28.028486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:28.041521 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.042345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.042985 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:28.043892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.044518 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.045321 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:28.075686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:30:28.076140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:30:28.076628 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:30:28.077201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:30:28.077859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:30:28.089903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.090375 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.091189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:28.095417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.096085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.103005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:28.133440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Getting quotas for project 81144c673e7b44dcb3b0f700ee628719. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:30:28.137831 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Getting quotas for user bbcc7dcbf32b4048879bbc77e0998b52 and project 81144c673e7b44dcb3b0f700ee628719. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:30:28.150480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:30:28.153004 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:30:28.222549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8d154fba-5ee7-46f7-bca2-cce1c4c3bd59 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.967166 Oct 12 02:30:28.223624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 84/164] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:30:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 969 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:30:28.228232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:28.228935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:28.235550 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.236118 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.236832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:28.238233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.238860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.239810 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:28.262554 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:28.275136 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a41eba22-eb24-4487-b9b6-7bdbe51c853e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1554 microversion: 2.1 time: 0.251535 Oct 12 02:30:28.286618 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 81/165] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:28 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1554 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:28.309429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b2ee97b3-22ca-4b1e-bea3-9345d36a9f0c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:28.310274 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b2ee97b3-22ca-4b1e-bea3-9345d36a9f0c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:28.374059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b2ee97b3-22ca-4b1e-bea3-9345d36a9f0c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:28.385555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b2ee97b3-22ca-4b1e-bea3-9345d36a9f0c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1219 microversion: 2.1 time: 0.095555 Oct 12 02:30:28.386151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 82/166] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:28 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:28.689790 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:28.704943 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-466c9332-df0a-42b3-9f89-5cf4682c5f24 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.476730 Oct 12 02:30:28.705578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 85/167] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:28 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:28.721888 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:28.723439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:28.735062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.750305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.750305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:28.751839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:28.755999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:28.757928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:29.303374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:29.303980 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:29.320540 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:29.320977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:29.321450 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:29.322152 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:29.322638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:29.323058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:29.343670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:29.352162 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3ba0e1c2-4a15-4971-937e-e1b121ee93c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.638578 Oct 12 02:30:29.352735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 83/168] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:28 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:29.411436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:29.412288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:29.418494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:29.419383 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:29.420001 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:29.430337 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:29.431230 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:29.431891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:29.711460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:29.719949 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-41e4e92f-5952-4ee1-8ca5-8dbd8c45c38d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1215 microversion: 2.1 time: 0.312794 Oct 12 02:30:29.721133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 84/169] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:29 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1215 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:29.736720 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:29.737989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:29.745409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:29.745895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:29.746319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:29.747827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:29.748280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:29.748919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:29.805946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:29.815293 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f9d640a6-a014-40f6-8cc3-8c299aa8c157 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1554 microversion: 2.1 time: 0.516252 Oct 12 02:30:29.816477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 86/170] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:29 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1554 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:29.954896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:29.972282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0a5579b1-2da3-44bb-93d5-299ea16214ad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.243940 Oct 12 02:30:29.978041 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 85/171] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:29 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:30.083654 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-270544536", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:30.150436 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] validate_networks() for None {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:30:30.380465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:30.381199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:30.387964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:30.388654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:30.389156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:30.390158 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:30.390613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:30.391344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:30.764686 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:30.781719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-de0a6be2-231f-4763-b1de-4f97ce35b665 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.406736 Oct 12 02:30:30.782736 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 86/172] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:30 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:30.795132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:30.795951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:30.808824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:30.809770 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:30.810675 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:30.811634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:30.813140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:30.813919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:30.918028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:30.927566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fb2a1d4d-8d09-46b6-a735-05f677c2ab76 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1277 microversion: 2.1 time: 0.143388 Oct 12 02:30:30.941339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 87/173] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:30 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1277 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:30.949531 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:30.950387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:30.957162 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:30.957658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:30.958093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:30.958789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:30.959242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:30.959669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.069855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:31.082639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0920a029-e7e6-4b58-a56e-5499d300de9c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1512 microversion: 2.1 time: 0.139945 Oct 12 02:30:31.083697 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 88/174] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:30 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1512 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:31.109974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:31.110646 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:31.118473 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.119237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.119889 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.121452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.122068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.122647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.169885 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:30:31.171682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:30:31.173946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:30:31.175186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:30:31.175975 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:30:31.188208 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.189112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.189860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.191748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.193403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.194187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.229875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Getting quotas for project 01036e8f062246c2a4fd6d398911cf1b. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:30:31.235876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Getting quotas for user e84163beca8c405bbdc3c07a15dec72b and project 01036e8f062246c2a4fd6d398911cf1b. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:30:31.253555 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:30:31.257955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:30:31.327513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-dbaf706e-bc71-4978-a987-fc64dd874a54 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.248204 Oct 12 02:30:31.337123 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 87/175] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:30:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 1256 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:30:31.356009 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-8135afd7-2dd5-429a-a652-7207573cef0c req-14b3eeb5-02a8-4ebe-b630-4068c3f8079e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a23966c6-4a75-42ef-8980-9d017342ee61", "name": "network-vif-deleted", "tag": "4926fd07-e534-4077-a727-12393bfe3a35"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:31.368789 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8135afd7-2dd5-429a-a652-7207573cef0c req-14b3eeb5-02a8-4ebe-b630-4068c3f8079e service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.369318 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8135afd7-2dd5-429a-a652-7207573cef0c req-14b3eeb5-02a8-4ebe-b630-4068c3f8079e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.369926 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8135afd7-2dd5-429a-a652-7207573cef0c req-14b3eeb5-02a8-4ebe-b630-4068c3f8079e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.385646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-8135afd7-2dd5-429a-a652-7207573cef0c req-14b3eeb5-02a8-4ebe-b630-4068c3f8079e service nova] Creating event network-vif-deleted:4926fd07-e534-4077-a727-12393bfe3a35 for instance a23966c6-4a75-42ef-8980-9d017342ee61 on hv-cinder-84914 Oct 12 02:30:31.392865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-8135afd7-2dd5-429a-a652-7207573cef0c req-14b3eeb5-02a8-4ebe-b630-4068c3f8079e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054448 Oct 12 02:30:31.394286 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 88/176] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:31.399703 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a3dd22e3-71b9-4a40-be3a-f629d88cf798 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:31.400571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a3dd22e3-71b9-4a40-be3a-f629d88cf798 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:31.412653 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:31.420672 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ac300a6c-5e42-460d-acce-2a07f285b467 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.335816 Oct 12 02:30:31.424237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 89/177] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:31 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:31.476676 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a3dd22e3-71b9-4a40-be3a-f629d88cf798 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:31.485553 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a3dd22e3-71b9-4a40-be3a-f629d88cf798 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1214 microversion: 2.1 time: 0.089805 Oct 12 02:30:31.491765 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 89/178] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:31 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1214 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:31.805388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:31.806492 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:31.815394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.816932 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.817684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.818752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.819479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.820214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.954401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:31.955220 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:31.959688 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:31.965389 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.966114 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.966778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.967669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:31.968314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:31.968760 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:31.970014 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-13a9d787-f130-47df-8217-aafd91f37796 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.169341 Oct 12 02:30:31.971165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 90/179] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:31 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:32.103163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:32.110872 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-696fc54d-813b-4850-bdfa-cb263ee7b8cf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.160789 Oct 12 02:30:32.111179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:32.111781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:32.125325 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 90/180] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:31 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:32.125467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:32.125467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:32.125467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:32.125467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:32.125755 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:32.125755 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:32.185729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:32.195702 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-98eb978f-e095-4229-94f3-5cd500e3d2e4 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1512 microversion: 2.1 time: 0.088217 Oct 12 02:30:32.196276 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 91/181] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:32 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1512 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:32.446976 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:32.447655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:32.453881 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:32.454364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:32.454794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:32.455544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:32.456568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:32.457176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:32.514877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:32.515681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:32.528666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:32.529416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:32.530089 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:32.533814 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:32.534505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:32.535125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:32.629686 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:32.649557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5690b370-f1a3-49b7-8ee6-4091e16e9f88 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1218 microversion: 2.1 time: 0.138609 Oct 12 02:30:32.650246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 92/182] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:32 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1218 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:32.744618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:32.757329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-df11f5c2-4b1b-420d-9361-bf3d41953285 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.314223 Oct 12 02:30:32.757995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 91/183] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:32 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:32.994238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:32.994840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:33.001985 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.002991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.003438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.004125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.004886 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.005451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.143015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:33.143960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:33.149930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.150283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.150886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.151734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.152344 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.152927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.249876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:33.261856 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b4f3fe1a-1b4e-4172-8b55-4503ca6cefee tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.271845 Oct 12 02:30:33.265264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 93/184] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:32 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:33.276125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:33.277497 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:33.294873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.295870 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.296477 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.298156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.305634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.306278 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.329352 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:33.339762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b97b9d4d-db72-4707-aea3-4495f441c057 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.200896 Oct 12 02:30:33.340911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 92/185] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:33 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:33.349032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.349679 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.350786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.351445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lazy-loading 'fault' on Instance uuid a23966c6-4a75-42ef-8980-9d017342ee61 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:30:33.414037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:33.422729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-82f50769-7879-43c7-beda-3b561ac9508e tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1350 microversion: 2.1 time: 0.156019 Oct 12 02:30:33.423536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 94/186] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:33 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1350 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:33.673828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:33.675947 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:33.684206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.687031 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.687740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.690460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.691165 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.691819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.759052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-c9e30493-08bd-41b6-84c6-d798a30d3fa8 req-5929848e-1de5-4bc0-a9d5-f54045dcd178 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "921f3627-5480-46cb-9810-fa6fcfde17ca", "tag": "57cbb2f7-a82b-48d3-8cf3-b111c66a8260"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:33.761528 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:33.769621 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-c9e30493-08bd-41b6-84c6-d798a30d3fa8 req-5929848e-1de5-4bc0-a9d5-f54045dcd178 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.770329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-c9e30493-08bd-41b6-84c6-d798a30d3fa8 req-5929848e-1de5-4bc0-a9d5-f54045dcd178 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.770944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-c9e30493-08bd-41b6-84c6-d798a30d3fa8 req-5929848e-1de5-4bc0-a9d5-f54045dcd178 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.773939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4e5e7196-3103-4853-b429-4bd0a9e86ba4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1274 microversion: 2.1 time: 0.104625 Oct 12 02:30:33.774836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 93/187] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:33 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1274 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:33.776399 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:33.800779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-c9e30493-08bd-41b6-84c6-d798a30d3fa8 req-5929848e-1de5-4bc0-a9d5-f54045dcd178 service nova] Creating event network-changed:57cbb2f7-a82b-48d3-8cf3-b111c66a8260 for instance 921f3627-5480-46cb-9810-fa6fcfde17ca on hv-cinder-84914 Oct 12 02:30:33.806773 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-c9e30493-08bd-41b6-84c6-d798a30d3fa8 req-5929848e-1de5-4bc0-a9d5-f54045dcd178 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051324 Oct 12 02:30:33.807863 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 95/188] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:33.812372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:33.813198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:33.820798 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.822142 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.822811 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:33.823678 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:33.824278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:33.824870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.122385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:34.130015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a5e65dea-dc1b-45bb-86c7-9838f2faebcf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.343143 Oct 12 02:30:34.131264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 94/189] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:33 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:34.293512 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:34.295200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:34.303444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.304177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.304987 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.305705 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.306299 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.306735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.366861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:34.367921 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:34.376018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.376935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.378092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.379042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.380147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.381203 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.595323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:34.603091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-09d0baf2-3889-402a-b4fe-847d4a8e20da tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.314301 Oct 12 02:30:34.603644 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 96/190] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:34 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:34.608069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:34.608662 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:34.615602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.616291 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.616747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.617445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.617903 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.618334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.643159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:34.652151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-09d1e65d-0f6c-4331-b458-d07111b22205 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.289515 Oct 12 02:30:34.653588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 95/191] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:34 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:34.653820 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.654415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.654853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.655305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lazy-loading 'fault' on Instance uuid a23966c6-4a75-42ef-8980-9d017342ee61 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:30:34.706264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:34.714652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b7cc7761-d66c-4932-8350-79d9118a2c5b tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1350 microversion: 2.1 time: 0.110013 Oct 12 02:30:34.715584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 97/192] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:34 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1350 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:34.801688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:34.802460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:34.808367 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.809108 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.809536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.810231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:34.810680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:34.811094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:34.939783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:34.950382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-211c076c-478f-4354-b2f8-90f06226d686 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1272 microversion: 2.1 time: 0.152938 Oct 12 02:30:34.951188 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 96/193] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:34 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1272 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:35.153769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:35.154607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:35.161375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.161855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.162448 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.163297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.163745 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.164160 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.324593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:35.332991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-03192b2b-8c79-4e69-986c-5c301cd4fb99 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.183031 Oct 12 02:30:35.333575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 98/194] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:35 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:35.626041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:35.626650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:35.633104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.633755 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.634338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.635311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.635759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.636169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.676750 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:35.677472 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:35.685538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.686034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.686464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.687477 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.687573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.688003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.781830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:35.793808 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d91eaae8-6b61-449f-a44a-4c893f009245 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.171629 Oct 12 02:30:35.794631 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 97/195] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:35 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:35.806975 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:35.807708 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:35.814479 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.815607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.816419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.817175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.817626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.818615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.851986 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.852476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.853049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.853511 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lazy-loading 'fault' on Instance uuid a23966c6-4a75-42ef-8980-9d017342ee61 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:30:35.858486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:35.873295 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b046418b-2148-4058-ba49-37066bf3fa28 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.197624 Oct 12 02:30:35.876397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 99/196] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:35 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:35.911545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47f30e76f6894e5f9c06f71adde020e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60e8d0fb43eb419ba1c5eb830cc89ee3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:35.920448 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0d12579a-2b9f-42e9-ac19-6fc281a55f43 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 200 len: 1350 microversion: 2.1 time: 0.124479 Oct 12 02:30:35.921048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 98/197] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:35 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 1350 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:35.977553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:35.978397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:35.984500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.985045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.985475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:35.986186 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:35.986628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:35.987620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:36.133383 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:36.147219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a849f87b-731f-4406-8058-2d3f797226ce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.172884 Oct 12 02:30:36.147596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 100/198] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:35 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:36.358423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:36.359283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:36.372941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:36.373627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:36.374220 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:36.376286 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:36.377211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:36.377899 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:36.819866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:36.824951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:36.849872 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:36.850625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:36.852438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:36.853182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:36.853787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:36.857528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:36.880547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:36.891392 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c3b1641b-cbc1-4730-b28e-3b06be172490 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.538047 Oct 12 02:30:36.906016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 99/199] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:36 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:36.915963 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:36.916970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:36.924917 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:36.926532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:36.927171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:36.928071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:36.928746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:36.929505 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.057958 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:37.069845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-213e9f74-f201-48d6-919c-05f7fc03e046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.255999 Oct 12 02:30:37.083371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 101/200] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:36 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:37.088638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:37.089283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:37.095788 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.096393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.100939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.100939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.100939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.100939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.120035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] HTTP exception thrown: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Request method failure captured: Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 HTTP/1.1 Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaNCVEFJsxkmxsTEf7RT8uFk1oJIbqwRyz3qO0uW_7OeOJFhEJTGH6yZwt_c5f_YRFNqEAraCJ6gz_7i17K4r4zo32w3Kb4_qxHOBPcjAJIhoMDHrcJxbAJQG3n--sPXZwZgJ2oHb-dVDz6uKTbtpKzcVmNRyWIuZzbPKGbsftIdlc Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:30:37.128605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_60e8d0fb43eb419ba1c5eb830cc89ee3", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 47f30e76f6894e5f9c06f71adde020e6 Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.129853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'a23966c6-4a75-42ef-8980-9d017342ee61'} Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:37.130540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.132095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.133085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Returning 404 to user: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:37.133085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-612c465a-1cb0-458a-83ff-bc769fbc90a3 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 404 len: 111 microversion: 2.1 time: 0.046433 Oct 12 02:30:37.133085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 102/201] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:37 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:37.172391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:37.173031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:37.188481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.191714 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.193624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.194709 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.195179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.195602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.303629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:37.320902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-115f0381-c764-4e68-b63a-7e9297649ece tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.413415 Oct 12 02:30:37.321480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 100/202] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:36 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:37.330217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:37.334943 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:37.354613 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.355370 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.356268 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.357288 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.359608 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.360199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.390068 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] HTTP exception thrown: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Request method failure captured: Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61/os-volume_attachments HTTP/1.1 Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaNCVEFJsxkmxsTEf7RT8uFk1oJIbqwRyz3qO0uW_7OeOJFhEJTGH6yZwt_c5f_YRFNqEAraCJ6gz_7i17K4r4zo32w3Kb4_qxHOBPcjAJIhoMDHrcJxbAJQG3n--sPXZwZgJ2oHb-dVDz6uKTbtpKzcVmNRyWIuZzbPKGbsftIdlc Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:30:37.402414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_60e8d0fb43eb419ba1c5eb830cc89ee3", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 47f30e76f6894e5f9c06f71adde020e6 Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.403905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'server_id': 'a23966c6-4a75-42ef-8980-9d017342ee61'} Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:37.404574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:37.405640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:37.408130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Returning 404 to user: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:37.409801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5926fca2-a560-417b-b7e6-af5406b89e75 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.087137 Oct 12 02:30:37.410981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 101/203] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:30:37 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61/os-volume_attachments => generated 111 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:37.498920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:37.509584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-026e1c6c-1379-47e9-b413-b57ee9d48992 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.341020 Oct 12 02:30:37.512645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 103/204] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:37 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:37.575134 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-a49c2643-89bf-4240-ac63-190e85cf639d req-16ce0589-90fa-4a91-b782-555c29110eb6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7db1184c-8d98-4d39-b77c-fb92bb3fc950", "name": "network-vif-plugged", "status": "completed", "tag": "b820bded-6400-480e-810e-7da24b3b68f7"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:37.603498 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a49c2643-89bf-4240-ac63-190e85cf639d req-16ce0589-90fa-4a91-b782-555c29110eb6 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.604469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a49c2643-89bf-4240-ac63-190e85cf639d req-16ce0589-90fa-4a91-b782-555c29110eb6 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.605133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-a49c2643-89bf-4240-ac63-190e85cf639d req-16ce0589-90fa-4a91-b782-555c29110eb6 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.623747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-a49c2643-89bf-4240-ac63-190e85cf639d req-16ce0589-90fa-4a91-b782-555c29110eb6 service nova] Creating event network-vif-plugged:b820bded-6400-480e-810e-7da24b3b68f7 for instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 on hv-cinder-84914 Oct 12 02:30:37.629958 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-a49c2643-89bf-4240-ac63-190e85cf639d req-16ce0589-90fa-4a91-b782-555c29110eb6 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059767 Oct 12 02:30:37.632298 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 102/205] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:37.686609 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-4eb1a0d8-c36c-4eeb-b9ef-dabbe6eb6d7e req-d76ed614-f55c-403f-9a48-e0a344abf1fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1268f1e-0939-4370-963d-a06c09e8f4b1", "tag": "69ae80c5-527b-4a9b-80c2-30b01d238bf3"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:37.706642 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-4eb1a0d8-c36c-4eeb-b9ef-dabbe6eb6d7e req-d76ed614-f55c-403f-9a48-e0a344abf1fe service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.708618 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-4eb1a0d8-c36c-4eeb-b9ef-dabbe6eb6d7e req-d76ed614-f55c-403f-9a48-e0a344abf1fe service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.709371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-4eb1a0d8-c36c-4eeb-b9ef-dabbe6eb6d7e req-d76ed614-f55c-403f-9a48-e0a344abf1fe service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.731065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-4eb1a0d8-c36c-4eeb-b9ef-dabbe6eb6d7e req-d76ed614-f55c-403f-9a48-e0a344abf1fe service nova] Creating event network-changed:69ae80c5-527b-4a9b-80c2-30b01d238bf3 for instance a1268f1e-0939-4370-963d-a06c09e8f4b1 on hv-cinder-84914 Oct 12 02:30:37.741129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-4eb1a0d8-c36c-4eeb-b9ef-dabbe6eb6d7e req-d76ed614-f55c-403f-9a48-e0a344abf1fe service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058885 Oct 12 02:30:37.745465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 104/206] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:37.918935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:37.920197 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:37.929352 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.931309 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.931742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.932789 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:37.933237 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:37.935022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:37.958995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:38.094414 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:38.095456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:38.104942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.105744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.106212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.106999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.107505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.107918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.119229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:38.124377 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e0a1c038-e4fa-4ec3-b903-789bf03b5d29 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.209861 Oct 12 02:30:38.125285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 103/207] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:37 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:38.257372 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:38.266721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ef6994fc-eaf0-43ea-b965-ec3189ffe046 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.176381 Oct 12 02:30:38.267716 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 105/208] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:38 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:38.345530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:38.346587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:38.353581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.354331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.354905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.355756 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.356353 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.357191 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.522992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:38.531857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5e2b091c-f808-4419-b80d-42600085cf14 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.190717 Oct 12 02:30:38.532435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 104/209] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:38 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:38.535368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:38.535963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:38.539895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:38.545237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.545723 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.546155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.546842 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.547294 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.547711 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.695319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:38.705016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-68154504-42bd-4101-ad5e-c74755ce3ebb tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.173295 Oct 12 02:30:38.706135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 106/210] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:38 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:38.819871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:38.820357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:38.826524 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.827084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.827590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.828366 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.828869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.829289 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.844776 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] HTTP exception thrown: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Request method failure captured: Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: DELETE /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 HTTP/1.1 Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaNCVEFJsxkmxsTEf7RT8uFk1oJIbqwRyz3qO0uW_7OeOJFhEJTGH6yZwt_c5f_YRFNqEAraCJ6gz_7i17K4r4zo32w3Kb4_qxHOBPcjAJIhoMDHrcJxbAJQG3n--sPXZwZgJ2oHb-dVDz6uKTbtpKzcVmNRyWIuZzbPKGbsftIdlc Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:30:38.847482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_60e8d0fb43eb419ba1c5eb830cc89ee3", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 47f30e76f6894e5f9c06f71adde020e6 Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:38.848743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'a23966c6-4a75-42ef-8980-9d017342ee61'} Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:38.849415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:38.851072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:38.853207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Returning 404 to user: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:38.853207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7455b920-5b4d-4fb6-82c5-391d71ea6e55 tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "DELETE /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 404 len: 111 microversion: 2.1 time: 0.032799 Oct 12 02:30:38.853207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 105/211] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:30:38 2022] DELETE /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:38.857690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:38.858356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] [instance: a23966c6-4a75-42ef-8980-9d017342ee61] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:38.865612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.866120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.866553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:38.867446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:38.867891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:38.868539 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.025480 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] HTTP exception thrown: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Request method failure captured: Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 HTTP/1.1 Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaNCVEFJsxkmxsTEf7RT8uFk1oJIbqwRyz3qO0uW_7OeOJFhEJTGH6yZwt_c5f_YRFNqEAraCJ6gz_7i17K4r4zo32w3Kb4_qxHOBPcjAJIhoMDHrcJxbAJQG3n--sPXZwZgJ2oHb-dVDz6uKTbtpKzcVmNRyWIuZzbPKGbsftIdlc Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:30:39.027036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/60e8d0fb43eb419ba1c5eb830cc89ee3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_60e8d0fb43eb419ba1c5eb830cc89ee3", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 60e8d0fb43eb419ba1c5eb830cc89ee3 Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-DeleteServersTestJSON-2022453848 Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 47f30e76f6894e5f9c06f71adde020e6 Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-DeleteServersTestJSON-2022453848-project Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:39.028388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'a23966c6-4a75-42ef-8980-9d017342ee61'} Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:39.029197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. Oct 12 02:30:39.030516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:39.031572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] Returning 404 to user: Instance a23966c6-4a75-42ef-8980-9d017342ee61 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:39.031572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0796c45c-30e7-486c-b191-45d908a37a5c tempest-DeleteServersTestJSON-2022453848 tempest-DeleteServersTestJSON-2022453848-project] 10.222.0.23 "GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61" status: 404 len: 111 microversion: 2.1 time: 0.174804 Oct 12 02:30:39.032734 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 107/212] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:38 2022] GET /compute/v2.1/servers/a23966c6-4a75-42ef-8980-9d017342ee61 => generated 111 bytes in 179 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:39.147751 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:39.148355 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:39.155162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.155650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.156083 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.157025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.157483 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.158219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.294629 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:39.296947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:39.316366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.317349 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.318538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.319424 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.319880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.320456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.512302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:39.523140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ca38c4d7-f2e5-4026-9f78-f8f8d53fb88f tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1314 microversion: 2.1 time: 0.236164 Oct 12 02:30:39.523729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 108/213] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:39 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1314 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:39.539939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:39.553314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1116e8e6-eec7-4e1e-bd59-4cac6b9a187a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.409198 Oct 12 02:30:39.557603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 106/214] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:39 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:39.564097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:39.564764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:39.571190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.571681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.572297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.573027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.573474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.574940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.730659 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:39.731762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:39.743091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.744003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.744564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.745408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:39.745784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:39.746159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:39.746699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:39.755489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-94cb5f09-58af-41f6-ac2d-33e37af135a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.203752 Oct 12 02:30:39.761306 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 109/215] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:39 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:39.969103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:39.981146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0426aa99-62b8-4cdc-a383-23460a1befce tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.256946 Oct 12 02:30:39.982341 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 107/216] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:39 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:40.546253 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:40.547381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:40.555547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:40.556416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:40.557448 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:40.558331 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:40.559734 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:40.560336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:40.577293 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:40.578746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:40.595697 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:40.597213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:40.608518 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:40.610557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:40.611501 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:40.612802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:40.728333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:40.736447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c081a2c1-0f08-41f6-ab7c-94088153c0e1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1497 microversion: 2.1 time: 0.193853 Oct 12 02:30:40.757265 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 110/217] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:40 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:40.779084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:40.780297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:40.792195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:40.793648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:40.794336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:40.795514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:40.796215 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:40.797053 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:40.996777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:41.015121 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-34f09efe-5df4-4472-9a80-6f440d6ec89d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.240212 Oct 12 02:30:41.016311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 111/218] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:40 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:41.024679 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:41.025853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:41.036954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:41.040409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:41.041069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:41.042065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:41.042695 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:41.043376 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:41.045380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:41.055800 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-24beb14d-1d26-49c2-a525-5c9cea947444 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.482661 Oct 12 02:30:41.061713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 108/219] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:40 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:41.268119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:41.280379 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-098e601d-6513-4586-9db9-e09c8d468146 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.261891 Oct 12 02:30:41.282049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 112/220] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:41 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:41.627777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:41.628869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:41.638900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=4 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:41.640174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=14 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:41.643372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:41.644035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:41.646176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:41.647105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:41.647722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:41.648313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:41.695688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:30:41.870990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:41.880195 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-54655700-40a3-47f1-a0c6-c7166ee0c373 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "DELETE /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 204 len: 0 microversion: 2.1 time: 0.262659 Oct 12 02:30:41.882233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 109/221] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:30:41 2022] DELETE /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:30:41.893674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:41.895116 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:41.906374 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:41.917608 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:41.918105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:41.918623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:41.927956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:41.928907 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:41.931484 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:41.993293 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-2104dd2d-d733-4876-9f1a-bd64fbe3e415 req-5f9aa075-48b4-466f-ad82-a97852739129 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "921f3627-5480-46cb-9810-fa6fcfde17ca", "name": "network-vif-plugged", "status": "completed", "tag": "57cbb2f7-a82b-48d3-8cf3-b111c66a8260"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:42.011425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-2104dd2d-d733-4876-9f1a-bd64fbe3e415 req-5f9aa075-48b4-466f-ad82-a97852739129 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:42.013040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-2104dd2d-d733-4876-9f1a-bd64fbe3e415 req-5f9aa075-48b4-466f-ad82-a97852739129 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:42.013699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-2104dd2d-d733-4876-9f1a-bd64fbe3e415 req-5f9aa075-48b4-466f-ad82-a97852739129 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:42.045728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-2104dd2d-d733-4876-9f1a-bd64fbe3e415 req-5f9aa075-48b4-466f-ad82-a97852739129 service nova] Creating event network-vif-plugged:57cbb2f7-a82b-48d3-8cf3-b111c66a8260 for instance 921f3627-5480-46cb-9810-fa6fcfde17ca on hv-cinder-84914 Oct 12 02:30:42.057970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-2104dd2d-d733-4876-9f1a-bd64fbe3e415 req-5f9aa075-48b4-466f-ad82-a97852739129 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068978 Oct 12 02:30:42.059046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 110/222] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:42.066210 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:42.073844 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:42.092792 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:42.094077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:42.094793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:42.096479 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:42.097193 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:42.097835 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:42.356840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:42.368938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:42.369938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e24b8806-f66e-4c65-834b-726f3a2426a6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1503 microversion: 2.1 time: 0.483154 Oct 12 02:30:42.371548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 113/223] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:41 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1503 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:42.380152 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:42.381034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:42.384032 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d76d479a-52c6-4f65-9778-39a62c4b1f54 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.321892 Oct 12 02:30:42.384617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 111/224] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:42 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:42.400480 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:42.401374 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:42.401964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:42.402112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:42.402788 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:42.403176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:42.403780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:42.404351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:42.410067 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:42.413931 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:42.415738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:42.418365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:42.419310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:42.422080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:42.424975 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:42.662014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:42.671752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-90e36c8d-6846-41f7-b9d4-c190dd09e711 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.278715 Oct 12 02:30:42.672443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 112/225] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:42 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:42.711920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:42.719924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0c949711-493d-4d06-accd-d1b426b8ebfb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.347165 Oct 12 02:30:42.720676 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 114/226] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:42 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:43.396773 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:43.397386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:43.407851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:43.408977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:43.419812 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:43.420302 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:43.421823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:43.422780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:43.423462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:43.423591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:43.423956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:43.426019 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:43.426465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:43.427303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:43.433992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:43.434657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:43.759715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:43.771256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-094be669-11e5-4b2a-b4e8-d0430ba86a4e tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1503 microversion: 2.1 time: 0.380526 Oct 12 02:30:43.771924 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 113/227] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:43 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1503 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:43.777611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:43.778510 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:43.785156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:43.785778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:43.786308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:43.787236 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:43.788274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:43.788274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:43.961576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:43.977266 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-098a7e81-419e-4549-a386-6657269624d1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.573464 Oct 12 02:30:43.981432 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 115/228] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:43 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:43.990519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:43.992175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:44.011422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:44.012062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:44.012540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:44.013306 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:44.013762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:44.014183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:44.038909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:44.056839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5b85ec83-c5fe-4e8f-999d-c3d9dc6cd58c tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.282817 Oct 12 02:30:44.058012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 114/229] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:43 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:44.369809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:44.390242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cd583f43-b5b3-4ed8-a270-e948c765a6a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.407288 Oct 12 02:30:44.390864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 116/230] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:43 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:44.735657 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-d7a2eda3-234c-4aa8-be8e-083d3ac5158e req-ed4d6997-36c9-43e3-ba00-1e07f38a9bd3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7db1184c-8d98-4d39-b77c-fb92bb3fc950", "name": "network-vif-deleted", "tag": "b820bded-6400-480e-810e-7da24b3b68f7"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:44.751117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-d7a2eda3-234c-4aa8-be8e-083d3ac5158e req-ed4d6997-36c9-43e3-ba00-1e07f38a9bd3 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:44.751774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-d7a2eda3-234c-4aa8-be8e-083d3ac5158e req-ed4d6997-36c9-43e3-ba00-1e07f38a9bd3 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:44.752495 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-d7a2eda3-234c-4aa8-be8e-083d3ac5158e req-ed4d6997-36c9-43e3-ba00-1e07f38a9bd3 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:44.766712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-d7a2eda3-234c-4aa8-be8e-083d3ac5158e req-ed4d6997-36c9-43e3-ba00-1e07f38a9bd3 service nova] Creating event network-vif-deleted:b820bded-6400-480e-810e-7da24b3b68f7 for instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 on hv-cinder-84914 Oct 12 02:30:44.774469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-d7a2eda3-234c-4aa8-be8e-083d3ac5158e req-ed4d6997-36c9-43e3-ba00-1e07f38a9bd3 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043551 Oct 12 02:30:44.777772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 115/231] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:44.794248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:44.797042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:44.805940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:44.806442 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:44.806880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:44.807583 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:44.808230 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:44.808655 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:44.914068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:44.923069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cedf0f9b-6c03-46c9-8ffd-add260c3e612 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1461 microversion: 2.1 time: 0.132700 Oct 12 02:30:44.924771 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 117/232] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:44 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1461 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:45.003163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:45.007557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:45.013946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.014428 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.014861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.015561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.016042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.016460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.084184 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:45.085413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:45.091399 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.092039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.092785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.094081 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.094543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.094970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.255218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:45.266477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c062f164-c381-4578-b911-e6ca9e1095c5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1319 microversion: 2.1 time: 0.266940 Oct 12 02:30:45.267311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 116/233] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:44 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1319 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:45.272569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:45.287096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-67f0f382-afe9-4dc4-af8b-1381ab34c99f tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.209379 Oct 12 02:30:45.288155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 118/234] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:45 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:45.413142 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:45.414157 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:45.427505 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.428145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.428435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.430001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.430732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.431385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.631681 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:45.641483 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-403ffb1b-5e7c-44cc-892b-8f86919e08b8 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.232417 Oct 12 02:30:45.643484 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 117/235] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:45 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:45.737605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-ee6493e4-f0d3-41cf-9425-61cda208c74b req-62fc6437-7866-424b-ab64-8c88edc68aa6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1268f1e-0939-4370-963d-a06c09e8f4b1", "name": "network-vif-plugged", "status": "completed", "tag": "69ae80c5-527b-4a9b-80c2-30b01d238bf3"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:45.762035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-ee6493e4-f0d3-41cf-9425-61cda208c74b req-62fc6437-7866-424b-ab64-8c88edc68aa6 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.762532 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-ee6493e4-f0d3-41cf-9425-61cda208c74b req-62fc6437-7866-424b-ab64-8c88edc68aa6 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.762968 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-ee6493e4-f0d3-41cf-9425-61cda208c74b req-62fc6437-7866-424b-ab64-8c88edc68aa6 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.787559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-ee6493e4-f0d3-41cf-9425-61cda208c74b req-62fc6437-7866-424b-ab64-8c88edc68aa6 service nova] Creating event network-vif-plugged:69ae80c5-527b-4a9b-80c2-30b01d238bf3 for instance a1268f1e-0939-4370-963d-a06c09e8f4b1 on hv-cinder-84914 Oct 12 02:30:45.795944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-ee6493e4-f0d3-41cf-9425-61cda208c74b req-62fc6437-7866-424b-ab64-8c88edc68aa6 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063556 Oct 12 02:30:45.797589 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 119/236] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:45.946856 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:45.947480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:45.956037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.956687 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.957125 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:45.958581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:45.959279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:45.959689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.002500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:46.003058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:46.003654 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.004141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lazy-loading 'fault' on Instance uuid 7db1184c-8d98-4d39-b77c-fb92bb3fc950 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:30:46.053952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:46.062241 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0388e332-3073-4010-a8e6-23410e1cae12 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1304 microversion: 2.1 time: 0.119190 Oct 12 02:30:46.062555 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 118/237] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:45 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1304 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:46.298229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:46.298812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:46.311279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:46.311960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:46.322107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:46.322606 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:46.322827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:46.323042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.323880 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:46.324341 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:46.324847 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.331309 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:46.331750 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.332474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:46.332962 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:46.333386 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.513440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:46.523311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-13321f1a-beb1-4bb3-bbfe-375d79b6d4a4 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.216038 Oct 12 02:30:46.539982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 119/238] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:46 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:46.624504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:46.648176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4250820c-1610-462e-a602-a184942e3e51 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1507 microversion: 2.1 time: 0.360359 Oct 12 02:30:46.651412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 120/239] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:46 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1507 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:46.670645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:46.671508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:46.681104 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:46.682087 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:46.682561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.683755 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:46.684217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:46.684954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:46.787777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-87738a21-edea-4fe9-b334-a25e5f043750 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Action: 'create', calling method: .version_select at 0x7f59d39a81f0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1995053884"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:46.924181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:46.933725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0bc48bf5-f7eb-4a8d-807f-6cdcfc340d61 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.271404 Oct 12 02:30:46.934465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 121/240] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:46 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:47.085438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:47.086044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:47.106445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:47.106948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:47.107378 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:47.113150 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:47.115880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:47.117464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:47.218779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:47.220384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:47.222055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:47.222251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-87738a21-edea-4fe9-b334-a25e5f043750 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.669137 Oct 12 02:30:47.222946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 120/241] 10.222.0.23 () {60 vars in 1155 bytes} [Wed Oct 12 02:30:46 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 670 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 12 02:30:47.223598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lazy-loading 'fault' on Instance uuid 7db1184c-8d98-4d39-b77c-fb92bb3fc950 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:30:47.235580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-79cd157b-5639-4cbf-ac22-aab46b640c33 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37a61f0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:47.285586 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:47.293508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0aa54e59-754e-454f-a8ac-7979435edbd6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 200 len: 1304 microversion: 2.1 time: 0.211853 Oct 12 02:30:47.294292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 122/242] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:47 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 1304 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:47.313878 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-79cd157b-5639-4cbf-ac22-aab46b640c33 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e" status: 200 len: 345 microversion: 2.1 time: 0.081429 Oct 12 02:30:47.315637 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 121/243] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:47 2022] GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e => generated 345 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:47.554328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:47.554932 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:47.567341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:47.569161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:47.569775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:47.570501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:47.570970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:47.571453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:47.708071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:47.715706 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4ac0f56a-cae6-45a5-be25-8a22404af499 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1314 microversion: 2.1 time: 0.172538 Oct 12 02:30:47.716263 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 123/244] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:47 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:47.957667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:47.958168 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:47.965194 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:47.965682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:47.966117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:47.966817 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:47.967269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:47.967684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:47.972146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:48.102908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:48.111481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-37aab286-a118-4a27-a3c1-29e55e278169 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.158482 Oct 12 02:30:48.112056 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 122/245] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:47 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:48.321262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:48.321262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: 7db1184c-8d98-4d39-b77c-fb92bb3fc950] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:48.324549 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:48.325077 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:48.325512 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:48.326225 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:48.326678 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:48.327129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:48.333799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b91a119e-6352-40a3-945d-d41e4617ffa3 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37b5160>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:48.341207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] HTTP exception thrown: Instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 could not be found. Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Request method failure captured: Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 HTTP/1.1 Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRia19hWVtTlI4MgxCV6LngRarMtx8ZaHTzjVz4Omsd7G61CEKApQKHyXDj40cEfaSTBsY45syawuLjmCsVLjiUTpv_3EY75Z2Cj3On7N-HzNSHOGtsE-TXWlS5nblPmWIN81aQdZqtrz-oJDzi6p-kKG7EJVkvHbsaeqBk5wn9FcR6A Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 177af31ad03c40e6b2a4a72c16451b5d Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesActionsTest-1485306525 Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:30:48.342845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/177af31ad03c40e6b2a4a72c16451b5d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/177af31ad03c40e6b2a4a72c16451b5d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/177af31ad03c40e6b2a4a72c16451b5d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_177af31ad03c40e6b2a4a72c16451b5d", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesActionsTest-1485306525 Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 177af31ad03c40e6b2a4a72c16451b5d Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesActionsTest-1485306525 Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesActionsTest-1485306525-project Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: a2ce151a58444f84b06a6e10cdebf1c3 Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesActionsTest-1485306525-project Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 could not be found. Oct 12 02:30:48.344200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '7db1184c-8d98-4d39-b77c-fb92bb3fc950'} Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 could not be found. Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:48.345084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 could not be found. Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 could not be found. Oct 12 02:30:48.346410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:48.347443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Returning 404 to user: Instance 7db1184c-8d98-4d39-b77c-fb92bb3fc950 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:48.347443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4c174249-608b-4ffa-be69-8b597b8501c2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950" status: 404 len: 111 microversion: 2.1 time: 0.031506 Oct 12 02:30:48.356019 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 124/246] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:48 2022] GET /compute/v2.1/servers/7db1184c-8d98-4d39-b77c-fb92bb3fc950 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:48.391201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b91a119e-6352-40a3-945d-d41e4617ffa3 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e" status: 200 len: 346 microversion: 2.1 time: 0.061012 Oct 12 02:30:48.391899 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 123/247] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:48 2022] GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e => generated 346 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:48.405830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-54bf426c-0caf-4d40-910f-19d3dcdb525e tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Action: 'create', calling method: .version_select at 0x7f59d37dbd30>, body: {"snapshot": {"volume_id": "7441020c-ed3a-4aae-8aff-86667eb75f9e", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1704793777"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:48.463639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1257870580", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "networks": [{"uuid": "cc86f2f4-a79e-41f7-816d-bf9acbcc5b5c"}]}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:48.576912 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-54bf426c-0caf-4d40-910f-19d3dcdb525e tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.175919 Oct 12 02:30:48.577496 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 125/248] 10.222.0.23 () {60 vars in 1162 bytes} [Wed Oct 12 02:30:48 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:48.588909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bfe529fb-f9c8-40a8-8233-a00ea2dc5412 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37bd040>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:48.625487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bfe529fb-f9c8-40a8-8233-a00ea2dc5412 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 200 len: 287 microversion: 2.1 time: 0.040291 Oct 12 02:30:48.626409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 126/249] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:48 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 287 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:48.637604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b9563f20-f193-4a7d-90d9-b92466d840aa tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d3925ca0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:48.676400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b9563f20-f193-4a7d-90d9-b92466d840aa tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.042604 Oct 12 02:30:48.677040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 127/250] 10.222.0.23 () {58 vars in 1140 bytes} [Wed Oct 12 02:30:48 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:48.688605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-03e925d2-bfd2-45e0-83b9-09dd632d4455 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37dbe50>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:48.722616 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-03e925d2-bfd2-45e0-83b9-09dd632d4455 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 200 len: 287 microversion: 2.1 time: 0.038077 Oct 12 02:30:48.723182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 128/251] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:48 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 287 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:48.739845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:48.740442 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:48.746663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:48.747141 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:48.747566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:48.748347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:48.748807 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:48.749227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:48.884090 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:48.894853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7a5dfc3f-89dc-4b41-8c34-23d27eca1304 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1464 microversion: 2.1 time: 0.159728 Oct 12 02:30:48.895463 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 129/252] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:48 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1464 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:48.942375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:48.942874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:48.948953 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:48.949440 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:48.949874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:48.950680 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:48.951127 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:48.951559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:48.979612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:30:49.052746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0345ecf9-975a-4659-86af-51714f024150 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "DELETE /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 204 len: 0 microversion: 2.1 time: 0.114473 Oct 12 02:30:49.053349 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 130/253] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:30:48 2022] DELETE /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 12 02:30:49.062336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:49.063042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:49.069225 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:49.069791 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:49.070228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:49.070943 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:49.071478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:49.071903 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:49.208140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:49.215927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2cb7b263-ba9f-4d5d-b163-831849dad984 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1470 microversion: 2.1 time: 0.158012 Oct 12 02:30:49.216494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 131/254] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:49 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1470 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:49.221526 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:49.222127 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:49.227596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:49.228079 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:49.228511 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:49.229222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:49.229685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:49.230110 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:49.362079 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:49.370329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c21a2915-1f51-436d-92be-d67739425d4f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.152709 Oct 12 02:30:49.371062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 132/255] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:49 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:49.737763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4b662d42-5e2e-4723-a4e4-103fe7ac7dcd tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d39ba430>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:49.767815 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4b662d42-5e2e-4723-a4e4-103fe7ac7dcd tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 200 len: 287 microversion: 2.1 time: 0.034688 Oct 12 02:30:49.768387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 133/256] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:49 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 287 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:50.240203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:50.240846 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:50.246777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:50.247260 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:50.247695 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:50.248383 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:50.248870 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:50.249291 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:50.377794 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:50.386441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9addb07b-fb98-49bd-8bf4-b4e26e058ca7 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1470 microversion: 2.1 time: 0.150842 Oct 12 02:30:50.387030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 134/257] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:50 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1470 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:50.393552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:50.394215 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:50.404120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:50.404120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:50.404120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:50.404120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:50.404605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:50.404605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:50.408548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:30:50.543068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:50.551322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c67099c9-ebed-4e89-a412-ef3a08b87efe tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.161540 Oct 12 02:30:50.551899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 135/258] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:50 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:50.780773 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f58893f9-094b-417b-a26c-4f5586a53af5 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d38a3c10>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:50.813929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f58893f9-094b-417b-a26c-4f5586a53af5 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 200 len: 287 microversion: 2.1 time: 0.036945 Oct 12 02:30:50.815148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 136/259] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:50 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 287 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:51.410427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:51.411063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:51.416831 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:51.417312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:51.417742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:51.418423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:51.418872 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:51.419289 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:51.549063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:51.556620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-866c569b-6f59-4315-9968-0decfe81b51b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1470 microversion: 2.1 time: 0.150198 Oct 12 02:30:51.557204 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 137/260] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:51 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1470 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:51.574113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:51.574827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:51.580571 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:51.581106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:51.581547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:51.582249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:51.582711 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:51.583139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:51.717680 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:51.725837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1b2b201a-e184-42e0-a589-789e246562cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.155354 Oct 12 02:30:51.726400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 138/261] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:51 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:51.827350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7f68667f-39fd-452f-8abc-57ad15546aa3 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d39254c0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:51.856658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7f68667f-39fd-452f-8abc-57ad15546aa3 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 200 len: 287 microversion: 2.1 time: 0.033076 Oct 12 02:30:51.857228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 139/262] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:51 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:52.581155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:52.581786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:52.587597 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:52.588117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:52.588566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:52.589268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:52.589716 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:52.590129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:52.721683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:52.729255 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e4ba5a9b-4273-4ceb-8287-c25be88d1cea tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1470 microversion: 2.1 time: 0.153052 Oct 12 02:30:52.729809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 140/263] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:52 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1470 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:52.749482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:52.750105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:52.755933 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:52.756351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:52.756815 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:52.757527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:52.757984 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:52.758402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:52.894533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:52.902211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ddadd203-eed8-4439-9ac7-4492ad0babed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.156514 Oct 12 02:30:52.902862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 141/264] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:52 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:52.907126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7b80b073-ff72-4920-8a0e-1b82afcac24f tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37bd820>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:52.935812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7b80b073-ff72-4920-8a0e-1b82afcac24f tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 200 len: 288 microversion: 2.1 time: 0.031918 Oct 12 02:30:52.936367 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 142/265] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:52 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 288 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:52.947491 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-19ae4d8d-400d-40da-83c3-3918a759ff10 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d3925040>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:53.006933 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-19ae4d8d-400d-40da-83c3-3918a759ff10 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "DELETE /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 202 len: 0 microversion: 2.1 time: 0.063010 Oct 12 02:30:53.007452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 143/266] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 12 02:30:52 2022] DELETE /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 0 bytes in 64 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 12 02:30:53.015168 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e93a85e3-44d9-4061-acbc-5d75c30f2e08 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d39ba550>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:53.044640 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e93a85e3-44d9-4061-acbc-5d75c30f2e08 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 200 len: 287 microversion: 2.1 time: 0.033018 Oct 12 02:30:53.045241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 144/267] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:53 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:53.363018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] validate_networks() for [('cc86f2f4-a79e-41f7-816d-bf9acbcc5b5c', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:30:53.718724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:30:53.719219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:30:53.719774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:30:53.720244 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:30:53.720771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:30:53.730409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:53.730908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:53.731353 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:53.732062 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:53.732507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:53.732957 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:53.750001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Getting quotas for project 177af31ad03c40e6b2a4a72c16451b5d. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:30:53.754098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:53.754551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Getting quotas for user a2ce151a58444f84b06a6e10cdebf1c3 and project 177af31ad03c40e6b2a4a72c16451b5d. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:30:53.754704 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:53.761573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:53.762069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:53.762500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:53.763267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:53.763653 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:53.764079 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:53.766644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:30:53.769252 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:30:53.816905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6ead5655-e5a3-4c6d-b4f5-ef70c8dc0147 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 5.356888 Oct 12 02:30:53.823425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 124/268] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:30:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 5359 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 12 02:30:53.831475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c9a7b92f-7eac-49ab-9f31-42242ba87fda tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:53.832328 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c9a7b92f-7eac-49ab-9f31-42242ba87fda tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:53.889318 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c9a7b92f-7eac-49ab-9f31-42242ba87fda tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:53.896687 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c9a7b92f-7eac-49ab-9f31-42242ba87fda tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1214 microversion: 2.1 time: 0.071706 Oct 12 02:30:53.896857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:53.897633 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 125/269] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:53 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1214 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:53.905301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-884259cd-5190-40ca-aefc-86fc1c82853b tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1470 microversion: 2.1 time: 0.155279 Oct 12 02:30:53.906162 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 145/270] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:53 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1470 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:53.925620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:53.926223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:53.933136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:53.933617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:53.934041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:53.934748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:53.935198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:53.935599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:54.058675 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c2a9db4f-df8e-4d60-a31c-5eb1a44e968d tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d39c90d0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:54.075341 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:54.083686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-181bbc46-d027-4f44-a0c3-ee061deb7c81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.162117 Oct 12 02:30:54.084542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 126/271] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:53 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:54.098849 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-c2a9db4f-df8e-4d60-a31c-5eb1a44e968d tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] HTTP exception thrown: Snapshot 81b3b99a-15a5-4918-8b0e-cfa9d562e62a could not be found. Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c2a9db4f-df8e-4d60-a31c-5eb1a44e968d tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Request method failure captured: Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a HTTP/1.1 Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRibWmPSWT-36mtxzMdIIh1wyLGAjyxcDijavkvCR4vVeErsMCSv50Xh1kLKT6mnE_lIEh9x_WETxOA-8LI3XL63dLSvyuWMg9YXPD80mCHHJIcuNgv5Ba0y4xKS6cEhEoklFaGGK2AS1jplpSe3A9pjs7XHIctnHNf_Y6BzkLuFC7xw Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:30:54.103784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_91317c221b5a4457b7b9b4bdb1ab91f9", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 3425fb6461cc441099d4b4df26ffafc4 Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Snapshot 81b3b99a-15a5-4918-8b0e-cfa9d562e62a could not be found. Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '81b3b99a-15a5-4918-8b0e-cfa9d562e62a'} Oct 12 02:30:54.105049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 81b3b99a-15a5-4918-8b0e-cfa9d562e62a could not be found. (HTTP 404) (Request-ID: req-8dd8e6c4-ee77-4d73-8c56-e9dac18a14f9) Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:54.105797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:30:54.106869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 81b3b99a-15a5-4918-8b0e-cfa9d562e62a could not be found. Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 12 02:30:54.108557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 81b3b99a-15a5-4918-8b0e-cfa9d562e62a could not be found. Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 81b3b99a-15a5-4918-8b0e-cfa9d562e62a could not be found. Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c2a9db4f-df8e-4d60-a31c-5eb1a44e968d tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Returning 404 to user: Snapshot 81b3b99a-15a5-4918-8b0e-cfa9d562e62a could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:54.109930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c2a9db4f-df8e-4d60-a31c-5eb1a44e968d tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a" status: 404 len: 111 microversion: 2.1 time: 0.051030 Oct 12 02:30:54.110857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 146/272] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:30:54 2022] GET /compute/v2.1/os-snapshots/81b3b99a-15a5-4918-8b0e-cfa9d562e62a => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:54.121544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e628bd1a-770d-4ba3-a398-4051ad0d2ace tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37b5550>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:54.187544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e628bd1a-770d-4ba3-a398-4051ad0d2ace tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e" status: 202 len: 0 microversion: 2.1 time: 0.075326 Oct 12 02:30:54.188586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 127/273] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:30:54 2022] DELETE /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:30:54.200343 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2c02cf76-00d9-49f0-b37d-66a1dfd23edd tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37bd8b0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:54.254173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2c02cf76-00d9-49f0-b37d-66a1dfd23edd tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e" status: 200 len: 345 microversion: 2.1 time: 0.060038 Oct 12 02:30:54.254820 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 147/274] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:54 2022] GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e => generated 345 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:54.920213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:54.920861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:54.929906 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:54.930508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:54.937614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:54.938156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:54.938584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:54.939767 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:54.940213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:54.940627 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:54.946721 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:54.947208 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:54.947633 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:54.948318 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:54.948786 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:54.949202 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:55.036559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:55.051608 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-33e0cb7e-90a7-4348-8f04-ec3527274651 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1266 microversion: 2.1 time: 0.136080 Oct 12 02:30:55.052490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 128/275] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:54 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1266 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:55.102459 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:55.110222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d6849a9a-c677-4816-88be-e8fb8f9d9a21 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1470 microversion: 2.1 time: 0.184069 Oct 12 02:30:55.110905 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 148/276] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:54 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1470 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:55.111047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:55.112594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:55.120968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:55.121453 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:55.121883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:55.122568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:55.123016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:55.123428 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:55.269409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7a7e116c-1580-4b84-ae26-6a0b05452105 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37db700>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:55.303470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-7a7e116c-1580-4b84-ae26-6a0b05452105 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] HTTP exception thrown: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7a7e116c-1580-4b84-ae26-6a0b05452105 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Request method failure captured: Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e HTTP/1.1 Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRibWmPSWT-36mtxzMdIIh1wyLGAjyxcDijavkvCR4vVeErsMCSv50Xh1kLKT6mnE_lIEh9x_WETxOA-8LI3XL63dLSvyuWMg9YXPD80mCHHJIcuNgv5Ba0y4xKS6cEhEoklFaGGK2AS1jplpSe3A9pjs7XHIctnHNf_Y6BzkLuFC7xw Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:30:55.305544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_91317c221b5a4457b7b9b4bdb1ab91f9", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 3425fb6461cc441099d4b4df26ffafc4 Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '7441020c-ed3a-4aae-8aff-86667eb75f9e'} Oct 12 02:30:55.306940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. (HTTP 404) (Request-ID: req-9ae61ef4-b2ad-4750-842e-93833771d1aa) Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.307802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:30:55.309222 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 12 02:30:55.310328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7a7e116c-1580-4b84-ae26-6a0b05452105 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Returning 404 to user: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7a7e116c-1580-4b84-ae26-6a0b05452105 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e" status: 404 len: 109 microversion: 2.1 time: 0.041831 Oct 12 02:30:55.311685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 149/277] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:55 2022] GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e => generated 109 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:30:55.317840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a97d3e8a-b2d4-48f5-b09d-5fa3a38ff5be tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d36729d0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:55.325684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:55.335167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f806e586-ee03-4932-abda-0765a380187e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.228026 Oct 12 02:30:55.345901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 129/278] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:55 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:55.378185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-a97d3e8a-b2d4-48f5-b09d-5fa3a38ff5be tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] HTTP exception thrown: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a97d3e8a-b2d4-48f5-b09d-5fa3a38ff5be tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Request method failure captured: Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e HTTP/1.1 Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRibWmPSWT-36mtxzMdIIh1wyLGAjyxcDijavkvCR4vVeErsMCSv50Xh1kLKT6mnE_lIEh9x_WETxOA-8LI3XL63dLSvyuWMg9YXPD80mCHHJIcuNgv5Ba0y4xKS6cEhEoklFaGGK2AS1jplpSe3A9pjs7XHIctnHNf_Y6BzkLuFC7xw Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:30:55.380117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_91317c221b5a4457b7b9b4bdb1ab91f9", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 3425fb6461cc441099d4b4df26ffafc4 Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '7441020c-ed3a-4aae-8aff-86667eb75f9e'} Oct 12 02:30:55.381447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. (HTTP 404) (Request-ID: req-c519b88b-6c03-463b-a80e-56990da7f110) Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.385165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 12 02:30:55.386449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 12 02:30:55.387633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a97d3e8a-b2d4-48f5-b09d-5fa3a38ff5be tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Returning 404 to user: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a97d3e8a-b2d4-48f5-b09d-5fa3a38ff5be tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e" status: 404 len: 109 microversion: 2.1 time: 0.067981 Oct 12 02:30:55.388978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 150/279] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:30:55 2022] DELETE /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e => generated 109 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:55.389901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-5cf3e6b1-72f2-4929-843b-7c4d16a7a350 req-f3ec7da7-be30-4b5e-8842-28c0433e1c75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1268f1e-0939-4370-963d-a06c09e8f4b1", "name": "network-vif-deleted", "tag": "69ae80c5-527b-4a9b-80c2-30b01d238bf3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:55.392402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6dcc7841-61dd-4884-be6d-eafd122f1f54 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Calling method '.version_select at 0x7f59d37db820>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:55.397718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-5cf3e6b1-72f2-4929-843b-7c4d16a7a350 req-f3ec7da7-be30-4b5e-8842-28c0433e1c75 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:55.398193 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-5cf3e6b1-72f2-4929-843b-7c4d16a7a350 req-f3ec7da7-be30-4b5e-8842-28c0433e1c75 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:55.398624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-5cf3e6b1-72f2-4929-843b-7c4d16a7a350 req-f3ec7da7-be30-4b5e-8842-28c0433e1c75 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:55.416727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-5cf3e6b1-72f2-4929-843b-7c4d16a7a350 req-f3ec7da7-be30-4b5e-8842-28c0433e1c75 service nova] Creating event network-vif-deleted:69ae80c5-527b-4a9b-80c2-30b01d238bf3 for instance a1268f1e-0939-4370-963d-a06c09e8f4b1 on hv-cinder-84914 Oct 12 02:30:55.421838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-5cf3e6b1-72f2-4929-843b-7c4d16a7a350 req-f3ec7da7-be30-4b5e-8842-28c0433e1c75 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037164 Oct 12 02:30:55.422594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 130/280] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:55.435160 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-6dcc7841-61dd-4884-be6d-eafd122f1f54 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] HTTP exception thrown: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6dcc7841-61dd-4884-be6d-eafd122f1f54 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Request method failure captured: Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e HTTP/1.1 Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRibWmPSWT-36mtxzMdIIh1wyLGAjyxcDijavkvCR4vVeErsMCSv50Xh1kLKT6mnE_lIEh9x_WETxOA-8LI3XL63dLSvyuWMg9YXPD80mCHHJIcuNgv5Ba0y4xKS6cEhEoklFaGGK2AS1jplpSe3A9pjs7XHIctnHNf_Y6BzkLuFC7xw Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:30:55.437268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/91317c221b5a4457b7b9b4bdb1ab91f9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_91317c221b5a4457b7b9b4bdb1ab91f9", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 91317c221b5a4457b7b9b4bdb1ab91f9 Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1852654872 Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 3425fb6461cc441099d4b4df26ffafc4 Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1852654872-project Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '7441020c-ed3a-4aae-8aff-86667eb75f9e'} Oct 12 02:30:55.438648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. (HTTP 404) (Request-ID: req-b9dc7f01-ba80-48c3-ad2d-1674a6c2408e) Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.440050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:30:55.441279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 12 02:30:55.442456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6dcc7841-61dd-4884-be6d-eafd122f1f54 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] Returning 404 to user: Volume 7441020c-ed3a-4aae-8aff-86667eb75f9e could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6dcc7841-61dd-4884-be6d-eafd122f1f54 tempest-VolumesSnapshotsTestJSON-1852654872 tempest-VolumesSnapshotsTestJSON-1852654872-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e" status: 404 len: 109 microversion: 2.1 time: 0.055256 Oct 12 02:30:55.443780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 151/281] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:55 2022] GET /compute/v2.1/os-volumes/7441020c-ed3a-4aae-8aff-86667eb75f9e => generated 109 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:30:56.076091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:56.076821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:56.083530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:56.084022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:56.084455 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:56.085278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:56.086141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:56.086573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:56.135311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:56.135842 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:56.142258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:56.142751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:56.143182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:56.143876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:56.144329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:56.144769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:56.182461 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:56.182956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:56.183387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:56.183829 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lazy-loading 'fault' on Instance uuid a1268f1e-0939-4370-963d-a06c09e8f4b1 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:30:56.201141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:56.211003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0e97fbfa-797b-4767-932f-a2cad558ac87 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1272 microversion: 2.1 time: 0.139235 Oct 12 02:30:56.211582 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 131/282] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:56 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1272 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:56.249607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e84163beca8c405bbdc3c07a15dec72b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01036e8f062246c2a4fd6d398911cf1b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:56.259556 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fa9273b3-3ce5-4011-8bda-50377a7b698d tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 200 len: 1304 microversion: 2.1 time: 0.129182 Oct 12 02:30:56.260125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 152/283] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:56 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 1304 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:30:56.359493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:56.360103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:56.366581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:56.367073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:56.367508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:56.368206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:56.368686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:56.369130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:56.803780 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:56.815213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a4e54eaf-4386-44bd-b5be-db3605cfaa8f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.459171 Oct 12 02:30:56.816109 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 132/284] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:56 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:57.234149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:57.235509 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:57.243838 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:57.244436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:57.244972 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:57.245742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:57.246428 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:57.247064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:57.282415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:57.283716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] [instance: a1268f1e-0939-4370-963d-a06c09e8f4b1] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:57.292043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:57.292652 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:57.293218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:57.294022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:57.294505 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:57.295147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:57.312966 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] HTTP exception thrown: Instance a1268f1e-0939-4370-963d-a06c09e8f4b1 could not be found. Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Request method failure captured: Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 HTTP/1.1 Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRibDtbbcfyRg7VKyekAgB72HEWdDTcz0xqso3NSbtRhXqJmkkpOenYVdjiwTV6fdA4vHUdF-hRxUI2ApZ2yLg_1-a42KraYsLWT7hIv5tgB0KTE5mo3DTFVyJlG3K89XbooHnRwYwflGoWAAdLOj46m0reKdqOlZIcYf5lZrg28l3LM Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 01036e8f062246c2a4fd6d398911cf1b Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-VolumesNegativeTest-2147062932 Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:30:57.315094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/01036e8f062246c2a4fd6d398911cf1b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/01036e8f062246c2a4fd6d398911cf1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/01036e8f062246c2a4fd6d398911cf1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_01036e8f062246c2a4fd6d398911cf1b", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-VolumesNegativeTest-2147062932 Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 01036e8f062246c2a4fd6d398911cf1b Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-VolumesNegativeTest-2147062932 Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-VolumesNegativeTest-2147062932-project Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: e84163beca8c405bbdc3c07a15dec72b Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-VolumesNegativeTest-2147062932-project Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance a1268f1e-0939-4370-963d-a06c09e8f4b1 could not be found. Oct 12 02:30:57.316248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'a1268f1e-0939-4370-963d-a06c09e8f4b1'} Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1268f1e-0939-4370-963d-a06c09e8f4b1 could not be found. Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:30:57.316959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1268f1e-0939-4370-963d-a06c09e8f4b1 could not be found. Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1268f1e-0939-4370-963d-a06c09e8f4b1 could not be found. Oct 12 02:30:57.318010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:30:57.320570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] Returning 404 to user: Instance a1268f1e-0939-4370-963d-a06c09e8f4b1 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:30:57.321608 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2b759cb6-b6db-40ae-9037-f45e63fa5ae0 tempest-VolumesNegativeTest-2147062932 tempest-VolumesNegativeTest-2147062932-project] 10.222.0.23 "GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1" status: 404 len: 111 microversion: 2.1 time: 0.043257 Oct 12 02:30:57.322167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 133/285] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:57 2022] GET /compute/v2.1/servers/a1268f1e-0939-4370-963d-a06c09e8f4b1 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:30:57.433325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:57.442207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-332c1469-82cc-4414-b0ef-ab249717cda5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1314 microversion: 2.1 time: 0.212347 Oct 12 02:30:57.443888 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 153/286] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:57 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1314 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:57.722752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9d53842a-e6f2-4a8f-a01e-8a480551d394 tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Action: 'create', calling method: .version_select at 0x7f59d37d0790>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1670657737", "metadata": {"Type": "work"}}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:57.829702 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-0b2cf8c3-893f-4687-ae73-70613dd79543 req-3c87ecc9-595b-4c06-9804-63dcd363e90f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a28569a6-05f2-47d3-907b-5d2d28ea3379", "tag": "13169a7b-25b2-4358-a5b8-abaa69d4abac"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:30:57.852211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-0b2cf8c3-893f-4687-ae73-70613dd79543 req-3c87ecc9-595b-4c06-9804-63dcd363e90f service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:57.852851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-0b2cf8c3-893f-4687-ae73-70613dd79543 req-3c87ecc9-595b-4c06-9804-63dcd363e90f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:57.853561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-0b2cf8c3-893f-4687-ae73-70613dd79543 req-3c87ecc9-595b-4c06-9804-63dcd363e90f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:57.879608 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-0b2cf8c3-893f-4687-ae73-70613dd79543 req-3c87ecc9-595b-4c06-9804-63dcd363e90f service nova] Creating event network-changed:13169a7b-25b2-4358-a5b8-abaa69d4abac for instance a28569a6-05f2-47d3-907b-5d2d28ea3379 on hv-cinder-84914 Oct 12 02:30:57.884938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-0b2cf8c3-893f-4687-ae73-70613dd79543 req-3c87ecc9-595b-4c06-9804-63dcd363e90f service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059568 Oct 12 02:30:57.885638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 154/287] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:30:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:30:57.892941 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:57.893742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:57.900614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:57.901231 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:57.901774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:57.902580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:57.903571 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:57.904221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:58.065152 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:58.073229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a2585c1f-b8e0-419e-a09b-4e90f575088e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.185912 Oct 12 02:30:58.073756 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 155/288] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:57 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:58.171205 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9d53842a-e6f2-4a8f-a01e-8a480551d394 tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.579637 Oct 12 02:30:58.172937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 134/289] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 12 02:30:57 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 586 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 12 02:30:58.186604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1aca2dc3-8d85-4447-a77b-d58d37ed690d tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Calling method '.version_select at 0x7f59d389b940>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:58.256936 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1aca2dc3-8d85-4447-a77b-d58d37ed690d tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed" status: 200 len: 353 microversion: 2.1 time: 0.074222 Oct 12 02:30:58.257860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 156/290] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:58 2022] GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed => generated 353 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:58.465457 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:58.466060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:58.473314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:58.473793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:58.474581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:58.474891 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:58.475338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:58.475795 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:58.610655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:58.618699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9a64233e-b2a4-4158-b988-fd488b0af5d0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1314 microversion: 2.1 time: 0.157029 Oct 12 02:30:58.619253 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 135/291] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:58 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:59.096692 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:59.097560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:59.104042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:59.104531 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:59.105167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:59.105867 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:59.106322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:59.106743 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:59.245421 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:59.253600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-34828b24-e172-4456-a514-9f2b01f2e314 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.161515 Oct 12 02:30:59.254240 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 157/292] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:59 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:30:59.271106 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9806e93d-3034-4112-ab13-260cbe608ba9 tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Calling method '.version_select at 0x7f59d37b5dc0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:59.320008 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9806e93d-3034-4112-ab13-260cbe608ba9 tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed" status: 200 len: 354 microversion: 2.1 time: 0.052521 Oct 12 02:30:59.321240 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 136/293] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:59 2022] GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed => generated 354 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:59.334188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3c33072a-400c-4f9b-b619-02813e61802a tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Calling method '.version_select at 0x7f59d39c9040>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:59.384753 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3c33072a-400c-4f9b-b619-02813e61802a tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed" status: 200 len: 354 microversion: 2.1 time: 0.054264 Oct 12 02:30:59.385407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 158/294] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:59 2022] GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed => generated 354 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:59.404385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-527b2f71-512a-45b1-bf59-449a5289d51e tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Calling method '.version_select at 0x7f59d45ffdc0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:59.471119 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-527b2f71-512a-45b1-bf59-449a5289d51e tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed" status: 202 len: 0 microversion: 2.1 time: 0.070433 Oct 12 02:30:59.471691 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 137/295] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:30:59 2022] DELETE /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed => generated 0 bytes in 72 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:30:59.480071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b31ee972-e8e2-4a70-abb3-0eea6ea53ec5 tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Calling method '.version_select at 0x7f59d37bd700>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:59.534153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b31ee972-e8e2-4a70-abb3-0eea6ea53ec5 tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed" status: 200 len: 353 microversion: 2.1 time: 0.058229 Oct 12 02:30:59.534742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 159/296] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:30:59 2022] GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed => generated 353 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:30:59.644895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:30:59.644895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:30:59.651245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:59.651702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:59.652145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:59.652768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:30:59.653225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:30:59.653642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:30:59.788836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:30:59.798240 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-17b8f0cd-8259-4a29-b255-f9810c27a5d3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1314 microversion: 2.1 time: 0.158905 Oct 12 02:30:59.799103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 138/297] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:30:59 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:00.297961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:00.299558 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:00.325875 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:00.326401 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:00.326841 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:00.328924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:00.329506 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:00.329942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:00.514713 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:00.525156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5117ebcd-f255-46ea-902d-dac905715ccc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.237539 Oct 12 02:31:00.527311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 160/298] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:00 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:00.549682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b4919328-9298-48ac-8239-077b1c9432cd tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Calling method '.version_select at 0x7f59d37b5310>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:00.587305 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-b4919328-9298-48ac-8239-077b1c9432cd tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] HTTP exception thrown: Volume 20fa36ea-9076-4cc1-8662-9327bc7446ed could not be found. Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b4919328-9298-48ac-8239-077b1c9432cd tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Request method failure captured: Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed HTTP/1.1 Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRibhIc_00wYX8PGp28l0iUcyjHWvpraOO3enaUFhcG14R1wDhJsXKmNxNq_KUU-ZJZF-ZWqpLwXlY-Y7QmXDG9ZRZEzkES_-OmJls5Gmm5G-djx1daMMrvaB9KE4SbqntCPDqvNMOLD7LaVh-BWw8xgKPl-8oLjhvFk6KQcut0D6vK8 Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 2c97c0fb4b9d411eba5f2326ce32bf18 Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-VolumesGetTestJSON-1311517083 Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:31:00.591095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2c97c0fb4b9d411eba5f2326ce32bf18"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/2c97c0fb4b9d411eba5f2326ce32bf18"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/2c97c0fb4b9d411eba5f2326ce32bf18"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_2c97c0fb4b9d411eba5f2326ce32bf18", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-VolumesGetTestJSON-1311517083 Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 2c97c0fb4b9d411eba5f2326ce32bf18 Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-VolumesGetTestJSON-1311517083 Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-VolumesGetTestJSON-1311517083-project Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 70a1d657cea84d0d8fb285021ae35a2c Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-VolumesGetTestJSON-1311517083-project Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Volume 20fa36ea-9076-4cc1-8662-9327bc7446ed could not be found. Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '20fa36ea-9076-4cc1-8662-9327bc7446ed'} Oct 12 02:31:00.592568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 20fa36ea-9076-4cc1-8662-9327bc7446ed could not be found. (HTTP 404) (Request-ID: req-773e355f-864e-435e-b9e9-e7b5844e31e1) Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:31:00.593575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 12 02:31:00.597845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 20fa36ea-9076-4cc1-8662-9327bc7446ed could not be found. Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 12 02:31:00.601110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 20fa36ea-9076-4cc1-8662-9327bc7446ed could not be found. Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 20fa36ea-9076-4cc1-8662-9327bc7446ed could not be found. Oct 12 02:31:00.603390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:31:00.614203 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b4919328-9298-48ac-8239-077b1c9432cd tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] Returning 404 to user: Volume 20fa36ea-9076-4cc1-8662-9327bc7446ed could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:31:00.615604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b4919328-9298-48ac-8239-077b1c9432cd tempest-VolumesGetTestJSON-1311517083 tempest-VolumesGetTestJSON-1311517083-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed" status: 404 len: 109 microversion: 2.1 time: 0.069528 Oct 12 02:31:00.616423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 139/299] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:31:00 2022] GET /compute/v2.1/os-volumes/20fa36ea-9076-4cc1-8662-9327bc7446ed => generated 109 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:31:00.821164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:00.821766 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:00.827964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:00.828443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:00.828915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:00.829593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:00.830039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:00.830452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:00.981110 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:00.990023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5dd6b11a-c47a-46db-95d4-cf5e5aa5a9f2 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1314 microversion: 2.1 time: 0.172972 Oct 12 02:31:00.990592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 161/300] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:00 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:01.550307 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:01.550901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:01.557619 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:01.558096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:01.558519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:01.559192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:01.559633 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:01.560044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:01.703066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:01.712939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f9668091-1e2e-4593-acdf-343ef16cc01d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.167010 Oct 12 02:31:01.717578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 140/301] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:01 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:02.012264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:02.013535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:02.022212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:02.022867 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:02.023948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:02.024652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:02.025148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:02.025569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:02.275514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:02.286671 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-10555380-df96-4d22-983e-9c0e4457943a tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1314 microversion: 2.1 time: 0.278225 Oct 12 02:31:02.298319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 162/302] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:02 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1314 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:02.376883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-18e938cf-afab-4832-95f8-fe5031295f5b req-beff4482-af76-45d5-9fd5-316f7f583aae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a28569a6-05f2-47d3-907b-5d2d28ea3379", "name": "network-vif-plugged", "status": "completed", "tag": "13169a7b-25b2-4358-a5b8-abaa69d4abac"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:02.386843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-18e938cf-afab-4832-95f8-fe5031295f5b req-beff4482-af76-45d5-9fd5-316f7f583aae service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:02.387394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-18e938cf-afab-4832-95f8-fe5031295f5b req-beff4482-af76-45d5-9fd5-316f7f583aae service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:02.388073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-18e938cf-afab-4832-95f8-fe5031295f5b req-beff4482-af76-45d5-9fd5-316f7f583aae service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:02.400141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-18e938cf-afab-4832-95f8-fe5031295f5b req-beff4482-af76-45d5-9fd5-316f7f583aae service nova] Creating event network-vif-plugged:13169a7b-25b2-4358-a5b8-abaa69d4abac for instance a28569a6-05f2-47d3-907b-5d2d28ea3379 on hv-cinder-84914 Oct 12 02:31:02.405334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-18e938cf-afab-4832-95f8-fe5031295f5b req-beff4482-af76-45d5-9fd5-316f7f583aae service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032932 Oct 12 02:31:02.406239 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 141/303] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:31:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:31:02.740561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:02.741295 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:02.748643 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:02.750989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:02.751493 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:02.752208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:02.752669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:02.753129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:02.914673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:02.923391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2a373e9a-4e51-4d6e-bd68-918a346e4002 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.187483 Oct 12 02:31:02.924035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 163/304] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:02 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:03.316157 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:03.317966 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:03.328327 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:03.329638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:03.330323 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:03.331619 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:03.332269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:03.332878 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:03.524843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:03.536590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e7cb8059-ae08-4a37-9a30-563f69af0690 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1314 microversion: 2.1 time: 0.227036 Oct 12 02:31:03.536590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 142/305] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:03 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1314 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:03.946436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:03.947764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:03.953847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:03.954400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:03.954834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:03.955682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:03.956866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:03.957540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:04.110934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:04.120655 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9432f81c-12bb-463e-9ae3-0b2b5dc8ca16 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.177985 Oct 12 02:31:04.121334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 164/306] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:03 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:04.560727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:04.561434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:04.564003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:04.564761 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=8 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:04.570553 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:04.571151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:04.571632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:04.572386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:04.572933 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:04.573415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:04.734804 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:04.743828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bf623db3-d86b-4f13-ab9d-5cb44bd612c0 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1497 microversion: 2.1 time: 0.186961 Oct 12 02:31:04.744618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 143/307] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:04 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:05.145073 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:05.145673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:05.151915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:05.152390 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:05.152852 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:05.153531 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:05.154009 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:05.154430 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:05.158641 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:05.289096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:05.296779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-91a9685b-ec83-4a3f-a12f-f32ef0dfc479 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.156250 Oct 12 02:31:05.297329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 165/308] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:05 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:06.319825 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:06.320445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:06.326494 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:06.326977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:06.327421 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:06.328090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:06.328532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:06.328976 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:06.453545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:06.461291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6227efb4-c98f-4929-8a5f-60802ccff177 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.145299 Oct 12 02:31:06.461900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 144/309] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:06 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:07.483898 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:07.484503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:07.491126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:07.491605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:07.492032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:07.492727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:07.493183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:07.493600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:07.623357 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:07.630930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7a980335-e576-45e5-bcdd-64b08abda155 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.150850 Oct 12 02:31:07.631483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 166/310] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:07 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:08.438846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:08.439440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:08.445172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:08.445639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:08.446090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:08.446756 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:08.447196 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:08.447608 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:08.472378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:31:08.544189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-588543a3-4358-4ad2-a1ee-d31d5c8b9779 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "DELETE /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 204 len: 0 microversion: 2.1 time: 0.108822 Oct 12 02:31:08.544721 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 145/311] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:31:08 2022] DELETE /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:31:08.553357 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:08.553970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:08.560396 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:08.560922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:08.561356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:08.562038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:08.562509 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:08.562913 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:08.654052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:08.654745 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:08.661233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:08.661720 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:08.662159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:08.662842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:08.663290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:08.663997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:08.698471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:08.713269 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e776f74d-88f7-41cd-8691-6028c9e22ab1 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.163611 Oct 12 02:31:08.713859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 167/312] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:08 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:08.799029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:08.802587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:08.808396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1126c3fb-2b0d-4fbd-b126-7615351d2220 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.158468 Oct 12 02:31:08.808981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 146/313] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:08 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:09.736539 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:09.737172 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:09.743866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:09.744348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:09.744810 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:09.745494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:09.745948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:09.746370 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:09.831366 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:09.831971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:09.838267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:09.838782 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:09.839372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:09.840055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:09.840500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:09.840936 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:09.874113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:09.882395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b0b63417-7980-4597-96fc-9dd2bfadffbf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.149614 Oct 12 02:31:09.882960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 168/314] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:09 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:09.966477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:09.974049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8cee3347-05ff-4e6d-bdf8-1ca9cbc74574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.147388 Oct 12 02:31:09.974604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 147/315] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:09 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:10.905565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:10.906094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:10.912448 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:10.912962 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:10.913395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:10.914075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:10.914557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:10.914978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:10.997336 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:10.997938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:11.004491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:11.005006 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:11.005438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:11.006151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:11.006599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:11.007043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:11.040806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:11.049139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0d8c527f-165c-4603-819c-b78e84a5f8e6 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.147404 Oct 12 02:31:11.049697 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 169/316] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:10 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:11.139850 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:11.147431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ef8f8973-26ec-4aa5-b2cf-4c4315fd82c4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.153837 Oct 12 02:31:11.147988 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 148/317] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:10 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:12.072789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:12.073391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:12.075943 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=86,nova_api:UPDATE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:12.080896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:12.081376 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:12.081810 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:12.082492 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:12.082946 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:12.083363 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:12.170394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:12.171078 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:12.173657 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:UPDATE=2,nova_api:SELECT=84,nova_api:INSERT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:12.178998 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:12.179575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:12.180085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:12.180899 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:12.181423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:12.181908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:12.217091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:12.225543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8d529871-d05d-4cc3-b334-cde00668aa0b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.156597 Oct 12 02:31:12.226945 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 170/318] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:12 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:12.309418 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:12.317013 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6d3c0710-bcf0-4c63-a6d0-5943261cccd9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.150368 Oct 12 02:31:12.317570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 149/319] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:12 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:13.248680 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:13.249316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:13.256304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:13.256816 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:13.257251 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:13.257954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:13.258403 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:13.258828 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:13.340262 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:13.340889 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:13.347187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:13.347665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:13.348086 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:13.348855 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:13.349300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:13.349744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:13.386151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:13.394696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c62adaec-3526-46d6-976b-8f603223f2a4 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.149812 Oct 12 02:31:13.395267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 171/320] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:13 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:13.481310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:13.489212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-10879b02-59e5-42bc-9da3-81cc5ec5a18b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.152650 Oct 12 02:31:13.489782 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 150/321] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:13 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:14.417367 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:14.417968 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:14.424872 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:14.425342 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:14.425762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:14.426458 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:14.426900 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:14.427312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:14.512183 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:14.512892 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:14.519445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:14.520132 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:14.520573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:14.521294 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:14.521745 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:14.522171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:14.559623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:14.567412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bafc8afe-9192-4457-a64a-ee2c194b2a25 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.153864 Oct 12 02:31:14.567995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 172/322] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:14 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:14.649008 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:14.656575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b734bb19-3ba5-40bc-974f-f35f65669424 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.148251 Oct 12 02:31:14.657732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 151/323] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:14 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:15.591009 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:15.591625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:15.598746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:15.599220 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:15.599651 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:15.600334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:15.600821 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:15.601241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:15.681425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:15.683487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:15.688124 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:15.688609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:15.689069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:15.689748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:15.690197 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:15.690612 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:15.732358 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:15.741221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7fe17833-8d7c-44c5-b516-8b2986a1f4e5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.154001 Oct 12 02:31:15.741791 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 173/324] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:15 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:15.827270 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:15.835015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c887371e-2663-4367-b635-a10e7ccbafd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.157419 Oct 12 02:31:15.835571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 152/325] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:15 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:16.763874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:16.764477 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:16.770827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:16.771308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:16.771735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:16.772413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:16.772891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:16.773311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:16.858735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:16.859429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:16.866376 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:16.866939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:16.867461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:16.868215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:16.868763 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:16.869254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:16.906829 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:16.918032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-13ae8752-80f7-40fb-9792-e6a20e803075 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.157897 Oct 12 02:31:16.918844 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 174/326] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:16 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:17.001586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:17.009092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-41eb1366-9e7c-4a75-b93a-f41f3e2ab3dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.154201 Oct 12 02:31:17.009644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 153/327] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:16 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:17.942353 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:17.943035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:17.949457 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:17.949950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:17.950385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:17.951073 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:17.951528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:17.951950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:18.032263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:18.032892 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:18.038694 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:18.039167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:18.039595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:18.040269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:18.040724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:18.041156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:18.084254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:18.091925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-aa383756-7193-4594-8a1c-b2cadbfbb635 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.153899 Oct 12 02:31:18.092524 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 175/328] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:17 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:18.181403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:18.189519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1d2fca17-f55d-4083-b183-5f41bef95e7e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.161696 Oct 12 02:31:18.190086 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 154/329] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:18 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:18.328084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:18.329120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:18.334888 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:18.335367 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:18.335797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:18.336474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:18.337022 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:18.337502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:18.361180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1a8aeff1-1b68-4573-8a05-fe3558f4fb12 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/ips" status: 200 len: 150 microversion: 2.20 time: 0.036863 Oct 12 02:31:18.361737 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 176/330] 10.222.0.23 () {60 vars in 1289 bytes} [Wed Oct 12 02:31:18 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/ips => generated 150 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:31:19.122960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:19.123431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:19.129804 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:19.130294 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:19.130732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:19.131428 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:19.131905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:19.132783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:19.213368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:19.218229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:19.245851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:19.253329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:19.258728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:19.259723 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:19.262636 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:19.263119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:19.307241 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:19.318462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-14799fcb-b2cd-4c90-a9d5-8eb4525c8c91 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.198700 Oct 12 02:31:19.319205 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 155/331] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:19 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:19.430233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:19.438577 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-824a810b-e917-4beb-b0c2-5b22143e3138 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.229115 Oct 12 02:31:19.439142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 177/332] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:19 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:20.344943 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:20.345612 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:20.351629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.352114 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.352545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.353255 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.353704 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.354171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.462308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:20.462978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:20.470013 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.470601 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.471044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.471780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.472237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.472668 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.483061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:20.491210 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5ca02af0-8cae-4361-8877-d9f47c8c75c3 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.150143 Oct 12 02:31:20.491777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 156/333] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:20 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:20.496264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:20.496810 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:20.502339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.502821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.503248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.503933 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.504377 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.504814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.527692 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lazy-loading 'flavor' on Instance uuid 48eb4bd4-2680-4fc6-85c0-dae85fb740fa {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:31:20.604948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:20.613963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a037ec95-538f-4900-b4b0-5a0844cdadad tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.155538 Oct 12 02:31:20.614650 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 178/334] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:20 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:20.750138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8794dc5e-b7c9-4ca2-8fc2-ef68b5732f6d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/action" status: 202 len: 0 microversion: 2.20 time: 0.257157 Oct 12 02:31:20.751282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 157/335] 10.222.0.23 () {62 vars in 1319 bytes} [Wed Oct 12 02:31:20 2022] POST /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/action => generated 0 bytes in 259 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:31:20.759439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:20.760106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:20.766028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.766500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.766921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.767622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:20.768066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:20.768481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:20.897086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:20.905018 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:20.906948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:20.908074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c43f8b3f-1854-4174-8bc4-d41b5cb4c2ad tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.152499 Oct 12 02:31:20.908617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 179/336] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:20 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:21.514718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:21.515416 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:21.637430 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:21.637971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:21.645304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:21.645801 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:21.646206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:21.646895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:21.647339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:21.647751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:21.658822 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:21.659313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:21.659742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:21.660437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:21.660913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:21.661329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:21.781858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:21.789352 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-463b9a65-1f41-43d8-a261-1caa0c25423d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.155968 Oct 12 02:31:21.789915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 180/337] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:21 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:21.798931 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:21.807103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-74bd14ae-97ca-488a-88b4-0041aa04efaf tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.296622 Oct 12 02:31:21.807659 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 158/338] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:21 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:21.934446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:21.934980 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:21.940934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:21.941420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:21.941853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:21.942536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:21.942989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:21.943413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:22.066642 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:22.074693 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:22.076143 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:22.077334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-eb32bd69-df0e-4d19-a807-1fe771d6b844 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.146914 Oct 12 02:31:22.077889 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 181/339] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:21 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:22.811717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:22.812322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:22.818864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:22.819337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:22.819763 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:22.820433 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:22.820905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:22.821321 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:22.830393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:22.830995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:22.837308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:22.837786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:22.838223 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:22.838902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:22.839359 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:22.839779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:22.843643 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:22.949616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:22.957292 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0d587123-fa3f-483e-b684-7157d4fb6967 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.149415 Oct 12 02:31:22.957884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 159/340] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:22 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:22.967551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:22.975436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9f4527fb-b488-4fc1-9a11-750059e9245d tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.148912 Oct 12 02:31:22.976146 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 182/341] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:22 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:23.102508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:23.103033 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:23.108953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:23.109433 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:23.109861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:23.110537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:23.110984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:23.111403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:23.241336 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:23.249373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:23.251371 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:23.252514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-00c91f99-3549-4a86-95e1-2c1f9b615fd2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.153963 Oct 12 02:31:23.253082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 160/342] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:23 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:23.980198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:23.980837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:23.987828 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:23.988560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:23.989016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:23.989715 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:23.990158 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:23.990577 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:23.998935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:23.999597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:24.005403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:24.005882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:24.006313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:24.006987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:24.007437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:24.007855 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:24.025446 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:24.123010 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:24.131364 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-457fa8b0-8927-478c-a290-96aaaca40574 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.155004 Oct 12 02:31:24.132239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 183/343] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:23 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:24.137896 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:24.146164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-944326a7-0f4a-4049-8127-485a6041b212 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.151106 Oct 12 02:31:24.146711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 161/344] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:23 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:24.278145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:24.278765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:24.285147 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:24.285632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:24.286063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:24.286744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:24.287196 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:24.287613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:24.412877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:24.421068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:24.422508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:24.423648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-72ff9c51-a8ae-4732-a5dd-c58497313fe9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.149438 Oct 12 02:31:24.424202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 184/345] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:24 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:25.155206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:25.155818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:25.162122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:25.162609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:25.163023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:25.163707 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:25.164160 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:25.164580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:25.168635 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:25.169270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:25.176115 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:25.176600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:25.177066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:25.177753 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:25.178199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:25.178617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:25.302385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:25.307991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:25.310230 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1acdf24a-0f26-44f7-882d-6f15594ec8e9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.158855 Oct 12 02:31:25.310801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 162/346] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:25 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:25.316558 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2d58eeae-463e-4513-8c14-bfcae071afe8 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.151593 Oct 12 02:31:25.317138 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 185/347] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:25 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:25.448077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:25.448708 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:25.454607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:25.455117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:25.455545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:25.456247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:25.456714 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:25.457137 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:25.590195 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:25.608223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:25.610248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:25.611391 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-958c245a-da1f-466d-a5d0-da872c01ff25 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.167187 Oct 12 02:31:25.611927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 163/348] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:25 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:26.334219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:26.334830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:26.339060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:26.339669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:26.341311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:26.341799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:26.342234 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:26.342923 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:26.343375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:26.343795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:26.345483 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:26.345961 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:26.346390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:26.347071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:26.347513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:26.347926 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:26.472861 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:26.474023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:26.480545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9b2628dc-76a7-4569-8a59-9ef990634c09 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1522 microversion: 2.1 time: 0.150290 Oct 12 02:31:26.481140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 186/349] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:26 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:26.485157 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bc064b75-5ed6-4927-be96-d64ad506e715 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.146538 Oct 12 02:31:26.485157 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 164/350] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:26 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:26.638375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:26.639589 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:26.652927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:26.653421 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:26.653847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:26.654543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:26.655049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:26.655397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:26.719125 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-0c8e65c2-9bf2-42fa-a4f5-bd00834d4c52 req-e88eb683-86fb-4be6-b613-1e2dcfe01d8e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a28569a6-05f2-47d3-907b-5d2d28ea3379", "name": "network-vif-unplugged", "status": "completed", "tag": "13169a7b-25b2-4358-a5b8-abaa69d4abac"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:26.738079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-0c8e65c2-9bf2-42fa-a4f5-bd00834d4c52 req-e88eb683-86fb-4be6-b613-1e2dcfe01d8e service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:26.738745 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-0c8e65c2-9bf2-42fa-a4f5-bd00834d4c52 req-e88eb683-86fb-4be6-b613-1e2dcfe01d8e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:26.739550 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-0c8e65c2-9bf2-42fa-a4f5-bd00834d4c52 req-e88eb683-86fb-4be6-b613-1e2dcfe01d8e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:26.757953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-0c8e65c2-9bf2-42fa-a4f5-bd00834d4c52 req-e88eb683-86fb-4be6-b613-1e2dcfe01d8e service nova] Creating event network-vif-unplugged:13169a7b-25b2-4358-a5b8-abaa69d4abac for instance a28569a6-05f2-47d3-907b-5d2d28ea3379 on hv-cinder-84914 Oct 12 02:31:26.764029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-0c8e65c2-9bf2-42fa-a4f5-bd00834d4c52 req-e88eb683-86fb-4be6-b613-1e2dcfe01d8e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.049876 Oct 12 02:31:26.766363 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 165/351] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:31:26 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:31:26.817316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:26.826779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:26.828805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:26.830045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-02d3463a-9210-4d22-89b3-f562957f33ce tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.197114 Oct 12 02:31:26.830599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 187/352] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:26 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:27.505555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:27.506175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:27.506543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:27.506640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:27.512962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.513527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.513983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.514194 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.514694 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.514906 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.515122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.515379 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.515540 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.516161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.516771 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.517244 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.680977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:27.690931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6c2a947b-ead7-48a8-9855-f7a0cbbce972 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1502 microversion: 2.1 time: 0.189146 Oct 12 02:31:27.691499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 188/353] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:27 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1502 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:27.713400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:27.714072 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:27.721385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.722279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.722713 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.723415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.723867 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.724285 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.806944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:27.815661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-651b2dc5-3800-43bd-97af-abcdf3e3f7d5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1503 microversion: 2.1 time: 0.313741 Oct 12 02:31:27.816235 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 166/354] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:27 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1503 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:27.821206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-630382a2-826c-4f8a-97de-6200d68ce6ac req-6789b903-5945-4b84-81f5-b31b2d5db423 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a28569a6-05f2-47d3-907b-5d2d28ea3379", "name": "network-vif-deleted", "tag": "13169a7b-25b2-4358-a5b8-abaa69d4abac"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:27.831281 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-630382a2-826c-4f8a-97de-6200d68ce6ac req-6789b903-5945-4b84-81f5-b31b2d5db423 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.831944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-630382a2-826c-4f8a-97de-6200d68ce6ac req-6789b903-5945-4b84-81f5-b31b2d5db423 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.832374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-630382a2-826c-4f8a-97de-6200d68ce6ac req-6789b903-5945-4b84-81f5-b31b2d5db423 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.846624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-630382a2-826c-4f8a-97de-6200d68ce6ac req-6789b903-5945-4b84-81f5-b31b2d5db423 service nova] Creating event network-vif-deleted:13169a7b-25b2-4358-a5b8-abaa69d4abac for instance a28569a6-05f2-47d3-907b-5d2d28ea3379 on hv-cinder-84914 Oct 12 02:31:27.853953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-630382a2-826c-4f8a-97de-6200d68ce6ac req-6789b903-5945-4b84-81f5-b31b2d5db423 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036214 Oct 12 02:31:27.854558 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 167/355] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:31:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:31:27.859541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:27.860226 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:27.866665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.867319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.867895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.868767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:27.869515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:27.870072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:27.934791 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:27.945929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-af0694ed-b687-45e5-a0ce-c06086177a60 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1517 microversion: 2.1 time: 0.236280 Oct 12 02:31:27.947035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 189/356] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:27 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1517 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:28.049193 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:28.059559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:28.063054 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:28.064350 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8521e7b3-2b28-4d9e-a6e6-122289ba2734 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.208743 Oct 12 02:31:28.065145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 168/357] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:27 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:28.812543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2e4a5877-84d9-41fa-ace8-3cd3917d84e5", "device": "/dev/sdb"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:28.815528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:28.821887 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:28.822369 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:28.822798 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:28.823482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:28.823935 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:28.824352 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:28.843035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:28.844223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:28.850530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:28.851012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:28.851468 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:28.852344 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:28.852789 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:28.853212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:28.886386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:28.886868 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:28.887297 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:28.887735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lazy-loading 'fault' on Instance uuid a28569a6-05f2-47d3-907b-5d2d28ea3379 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:31:28.929251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:28.936909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c4d0f39b-c390-4a15-b8aa-8c46f5cd7e14 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1304 microversion: 2.1 time: 0.097575 Oct 12 02:31:28.938054 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 169/358] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:28 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1304 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:29.089616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:29.090220 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:29.102526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:29.102526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:29.102526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:29.102526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:29.102809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:29.102809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:29.231614 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:29.240718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:29.242208 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:29.243431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-678e9387-739c-4e87-ba08-7c86f6b96665 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.157593 Oct 12 02:31:29.244002 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 170/359] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:29 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:29.960774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:29.961411 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:29.967385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:29.967867 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:29.968301 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:29.969503 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:29.969959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:29.970378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:30.000662 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:30.001165 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:30.001603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:30.002040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lazy-loading 'fault' on Instance uuid a28569a6-05f2-47d3-907b-5d2d28ea3379 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:31:30.042503 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:30.052685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fbefedd2-3ac4-4ab4-8693-6ea7f62b17c5 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1304 microversion: 2.1 time: 0.095761 Oct 12 02:31:30.053279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 171/360] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:29 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1304 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:30.269245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:30.269884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:30.275744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:30.276229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:30.276664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:30.277372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:30.277833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:30.278256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:30.441827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:30.450560 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:30.452015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:30.453732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c487b707-17e0-4523-a9c8-c720e145c004 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.188975 Oct 12 02:31:30.454481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 172/361] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:30 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:31.075356 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:31.075969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:31.081859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:31.082331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:31.082762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:31.083439 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:31.083886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:31.084300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:31.116454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:31.117037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:31.117468 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:31.117896 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lazy-loading 'fault' on Instance uuid a28569a6-05f2-47d3-907b-5d2d28ea3379 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:31:31.162296 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a2ce151a58444f84b06a6e10cdebf1c3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '177af31ad03c40e6b2a4a72c16451b5d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:31.166053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f218bada-2d9f-4ab6-b22d-3c7e31d14f48 tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 200 len: 1304 microversion: 2.1 time: 0.094577 Oct 12 02:31:31.166603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 173/362] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:31 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 1304 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:31.480730 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:31.481870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:31.489635 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:31.490245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:31.490894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:31.492688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:31.493170 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:31.493589 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:31.530209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-314e6f45-5da7-4bae-9322-79d327a66267 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "POST /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.721489 Oct 12 02:31:31.533121 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 190/363] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:31:28 2022] POST /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca/os-volume_attachments => generated 194 bytes in 2725 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:31:31.765855 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:31.773771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:31.775206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:31.776343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-20abd909-52b8-4fb0-8fe2-f7b6b51f4946 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.300831 Oct 12 02:31:31.776915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 174/364] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:31 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:32.190230 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:32.190876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] [instance: a28569a6-05f2-47d3-907b-5d2d28ea3379] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:32.197659 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.198177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.198612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.199312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.199772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.200192 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.217381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] HTTP exception thrown: Instance a28569a6-05f2-47d3-907b-5d2d28ea3379 could not be found. Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Request method failure captured: Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 HTTP/1.1 Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRia19hWVtTlI4MgxCV6LngRarMtx8ZaHTzjVz4Omsd7G61CEKApQKHyXDj40cEfaSTBsY45syawuLjmCsVLjiUTpv_3EY75Z2Cj3On7N-HzNSHOGtsE-TXWlS5nblPmWIN81aQdZqtrz-oJDzi6p-kKG7EJVkvHbsaeqBk5wn9FcR6A Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 177af31ad03c40e6b2a4a72c16451b5d Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-VolumesActionsTest-1485306525 Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:31:32.219037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/177af31ad03c40e6b2a4a72c16451b5d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/177af31ad03c40e6b2a4a72c16451b5d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/177af31ad03c40e6b2a4a72c16451b5d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_177af31ad03c40e6b2a4a72c16451b5d", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-VolumesActionsTest-1485306525 Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 177af31ad03c40e6b2a4a72c16451b5d Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-VolumesActionsTest-1485306525 Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-VolumesActionsTest-1485306525-project Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: a2ce151a58444f84b06a6e10cdebf1c3 Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-VolumesActionsTest-1485306525-project Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance a28569a6-05f2-47d3-907b-5d2d28ea3379 could not be found. Oct 12 02:31:32.220451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'a28569a6-05f2-47d3-907b-5d2d28ea3379'} Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a28569a6-05f2-47d3-907b-5d2d28ea3379 could not be found. Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:31:32.221286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a28569a6-05f2-47d3-907b-5d2d28ea3379 could not be found. Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a28569a6-05f2-47d3-907b-5d2d28ea3379 could not be found. Oct 12 02:31:32.222546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:31:32.223754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] Returning 404 to user: Instance a28569a6-05f2-47d3-907b-5d2d28ea3379 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:31:32.223754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-de239c85-a4b1-4811-9fb7-bf50f586b21b tempest-VolumesActionsTest-1485306525 tempest-VolumesActionsTest-1485306525-project] 10.222.0.23 "GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379" status: 404 len: 111 microversion: 2.1 time: 0.035139 Oct 12 02:31:32.223754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 191/365] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:32 2022] GET /compute/v2.1/servers/a28569a6-05f2-47d3-907b-5d2d28ea3379 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:31:32.645218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:32.647548 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:32.653757 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.654349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.654882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.655632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.656903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.657560 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.685225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lazy-loading 'flavor' on Instance uuid 3d9e5223-f388-4f16-b15a-73c59e86de18 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:31:32.780034 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-47755e4e-5d00-4b68-81fc-60603dc41e06 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18/action" status: 200 len: 32 microversion: 2.1 time: 0.138280 Oct 12 02:31:32.785555 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 175/366] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:31:32 2022] POST /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18/action => generated 32 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:31:32.791445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:32.792125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:32.804904 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:32.805869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:32.809467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.810166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.810763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.811764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.812420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.812955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.814423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.814940 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.815845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.816557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:32.820159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:32.820953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:32.842247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:32.979593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:32.989596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2e69c08e-3534-43ac-988b-72459040892c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.202444 Oct 12 02:31:32.996748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 192/367] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:32 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:33.040883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:33.050962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:33.052570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:33.054057 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-54860d30-0c0b-49d0-867e-8200709f9bf4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.254955 Oct 12 02:31:33.054601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 176/368] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:32 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:34.015781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:34.016345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:34.022862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:34.023351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:34.023791 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:34.024481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:34.024997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:34.025390 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:34.084779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:34.085380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:34.091282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:34.091761 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:34.092186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:34.093033 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:34.094073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:34.094490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:34.157311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:34.164928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1b5c915c-8436-4009-b5e6-c24086969a81 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.153464 Oct 12 02:31:34.165489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 193/369] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:34 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:34.232506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:34.240799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:34.242225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:34.243361 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-15190956-0395-4866-a607-0a924a54059e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.162502 Oct 12 02:31:34.243898 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 177/370] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:34 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:35.187212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:35.187848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:35.194286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:35.194771 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:35.195210 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:35.195896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:35.196354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:35.196803 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:35.269067 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:35.269801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:35.277642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:35.277783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:35.278219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:35.278925 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:35.279378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:35.279796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:35.335888 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:35.344095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fcc24ccc-8c0f-44b9-8d02-da2cf6b91962 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.160673 Oct 12 02:31:35.344666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 194/371] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:35 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:35.439665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:35.447526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:35.448970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:35.450100 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b8292976-3c7e-44ab-b4d1-156ae4c005b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.184965 Oct 12 02:31:35.450646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 178/372] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:35 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:36.367561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:36.368166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:36.374660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:36.375140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:36.375573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:36.376266 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:36.376737 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:36.377192 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:36.478248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:36.479520 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:36.485564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:36.486052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:36.486522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:36.487206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:36.487654 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:36.488332 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:36.505951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:36.513502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a2acd14d-5ca4-42ad-95d3-9a69183658db tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.150265 Oct 12 02:31:36.514057 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 195/373] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:36 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:36.617742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:36.626200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:36.627631 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:36.628793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9b634afc-7288-4940-b1ae-8fa8e565cf4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.154829 Oct 12 02:31:36.629401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 179/374] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:36 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:37.537322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:37.537959 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:37.544257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:37.544767 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:37.545199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:37.545911 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:37.546929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:37.547347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:37.654566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:37.655162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:37.661578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:37.662058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:37.662486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:37.663169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:37.663618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:37.664036 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:37.673327 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:37.680941 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ddb75cba-52a5-4307-bf83-e3984c3e2ad5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.147363 Oct 12 02:31:37.681504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 196/375] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:37 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:37.793582 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:37.802733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:37.804174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:37.805403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1f157c96-2739-494f-b5b2-b7d430ca33f1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.154745 Oct 12 02:31:37.805965 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 180/376] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:37 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:38.703282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:38.703888 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:38.710269 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:38.710752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:38.711186 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:38.711895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:38.712349 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:38.712820 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:38.830604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:38.831199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:38.837057 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:38.837529 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:38.837952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:38.838625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:38.839065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:38.839477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:38.844197 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:38.853227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8ce88e26-f553-4385-87dd-c734633c21a6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.153778 Oct 12 02:31:38.853975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 197/377] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:38 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:38.971624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:38.979482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:38.980937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:38.982079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0b297aad-6340-4945-91b8-75ace4638f4c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.155494 Oct 12 02:31:38.982630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 181/378] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:38 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:39.877977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:39.878855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:39.884896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:39.885545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:39.885868 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:39.886560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:39.887012 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:39.887447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:39.892062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:40.007444 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:40.008052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:40.014180 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:40.015442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:40.016034 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:40.016909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:40.017508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:40.018066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:40.024149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:40.035110 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-aef30573-c4e2-49b7-91ff-a913d56c510c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.161480 Oct 12 02:31:40.036732 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 198/379] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:39 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:40.167525 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:40.175723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:40.177187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:40.178337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-34553103-0587-4427-a1ed-6268922cb839 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.174894 Oct 12 02:31:40.178894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 182/380] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:40 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:41.059467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:41.060148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:41.066573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:41.067060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:41.067500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:41.068187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:41.068647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:41.069098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:41.205857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:41.207805 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:41.223983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:41.224565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:41.225101 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:41.226345 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:41.227487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:41.228643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:41.246947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:41.260033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c56b9ef8-dd0a-4a10-b8a2-be4a41502c5b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.203623 Oct 12 02:31:41.263458 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 199/381] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:41 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:31:41.406048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:41.414994 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:41.417554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:41.419072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-65680539-c016-4f11-8fb3-3ed5db6e61a2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.219327 Oct 12 02:31:41.434515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 183/382] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:41 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:42.288093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:42.288936 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:42.291077 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=60 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:42.296997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:42.298467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:42.299304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:42.300339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:42.300944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:42.301675 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:42.451499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:42.453791 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:42.458786 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=76 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:42.469398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:42.470099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:42.470823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:42.474611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:42.476313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:42.477140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:42.596059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:42.619836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-de160789-0476-409e-bbca-961dfd05ef46 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.336376 Oct 12 02:31:42.620569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 200/383] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:42 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:42.716439 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:42.726206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:42.727853 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:42.729369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ba16f419-0c48-4499-b4c1-7d7c53aff11d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.284730 Oct 12 02:31:42.730073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 184/384] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:42 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:43.652574 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:43.653683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:43.661316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:43.662087 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:43.662761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:43.663744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:43.664416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:43.665080 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:43.758497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:43.760133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:43.775082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:43.775498 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:43.775941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:43.776721 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:43.777530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:43.778282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:43.824781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:43.833952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d8ce29a5-0c60-48fb-9701-29d9e2b19d0c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.185003 Oct 12 02:31:43.841216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 201/385] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:43 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:43.956230 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:43.964978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:43.966414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:43.967589 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9c59047e-8a90-4e59-b210-57e698ba19fd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.215435 Oct 12 02:31:43.980974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 185/386] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:43 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:44.857826 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:44.858507 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:44.865356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:44.865945 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:44.866469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:44.867421 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:44.867967 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:44.868598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:44.994163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:44.994754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:45.001362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:45.002053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:45.002667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:45.003535 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:45.004141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:45.004754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:45.023179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:45.032772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9eeb3aed-cc58-4d0f-83ea-76ff1a5a943a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.178818 Oct 12 02:31:45.049189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 202/387] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:44 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:45.162740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:45.171457 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:45.172919 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:45.174070 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b13c1c3f-36e8-4439-a993-c619b2280741 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.184405 Oct 12 02:31:45.174625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 186/388] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:44 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:46.055287 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:46.055902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:46.063493 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:46.063985 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:46.064417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:46.065174 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:46.065631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:46.066050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:46.200029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:46.200623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:46.207519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:46.208568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:46.209245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:46.210146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:46.210809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:46.211232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:46.251474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:46.261513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4987e0b4-8dbd-465c-8c6c-0575f1fcd33d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.209720 Oct 12 02:31:46.262211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 203/389] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:46 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:46.369734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:46.381192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:46.383011 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:46.384249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f1b55609-8725-42b3-afd4-f19d987e97f7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.188581 Oct 12 02:31:46.384913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 187/390] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:46 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:47.286992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:47.288433 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:47.296429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:47.297299 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:47.297743 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:47.298464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:47.298930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:47.299345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:47.419030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:47.419812 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:47.425972 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:47.426537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:47.427055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:47.427959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:47.428562 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:47.429154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:47.464665 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:47.495092 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-01626cc0-0f54-46e8-99a3-2dba6d138554 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.212652 Oct 12 02:31:47.499924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 204/391] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:47 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:47.584476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:47.595018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:47.596503 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:47.597881 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1f5011d7-a6a6-44ed-bd0d-1b91b5ab743e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.182874 Oct 12 02:31:47.598434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 188/392] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:47 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:48.519431 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:48.519991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:48.526617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:48.527089 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:48.527524 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:48.528223 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:48.528690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:48.529133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:48.628865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:48.630047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:48.636829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:48.637629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:48.638262 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:48.639149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:48.640335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:48.640974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:48.696863 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:48.705449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-eb8b188c-808d-424c-80a3-517d0bac98eb tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.189891 Oct 12 02:31:48.706055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 205/393] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:48 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:48.788960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:48.798088 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:48.799536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:48.801069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5e84501c-17c6-4a77-8888-d9e0bb7d4641 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.177522 Oct 12 02:31:48.801639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 189/394] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:48 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:49.728905 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:49.730246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:49.738083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:49.738566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:49.738995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:49.739713 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:49.740164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:49.740583 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:49.828571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:49.829885 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:49.837278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:49.838236 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:49.839028 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:49.839955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:49.840463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:49.840916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:49.894866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:49.902724 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9582bf80-3dc7-4d4e-ab8c-fe133d73824b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.177892 Oct 12 02:31:49.903304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 206/395] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:49 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:49.989973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:49.998402 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:50.000266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:50.001435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-127f6450-8a2e-4362-a05e-bc0f817b36ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1744 microversion: 2.20 time: 0.177800 Oct 12 02:31:50.002154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 190/396] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:49 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1744 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:50.926591 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:50.927834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:50.934866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:50.936002 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:50.937292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:50.938249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:50.938702 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:50.939340 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:51.028059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:51.029001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:51.038917 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:51.040295 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:51.041728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:51.042668 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:51.043270 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:51.043876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:51.163918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:51.173286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8640ad77-17be-4fcc-854b-c6e2676acb41 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.250774 Oct 12 02:31:51.177055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 207/397] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:50 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:51.360981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:51.372039 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:51.373915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:51.375389 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-69028f78-707a-4da3-8726-bc6251c1a4b3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1744 microversion: 2.20 time: 0.351690 Oct 12 02:31:51.375986 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 191/398] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:51 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1744 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:52.198972 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:52.200169 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:52.207126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:52.207931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:52.208627 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:52.210249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:52.211300 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:52.212394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:52.405359 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:52.405976 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:52.412409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:52.413205 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:52.413952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:52.414976 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:52.415703 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:52.416266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:52.424176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:52.437395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6de0b5c1-5d88-4699-bc74-afee758d1f49 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.243809 Oct 12 02:31:52.439613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 208/399] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:52 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:52.614923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:52.631128 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:52.632566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:52.633918 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1699da71-dae8-4c02-aa75-64e10e7cd887 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1744 microversion: 2.20 time: 0.232363 Oct 12 02:31:52.634533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 192/400] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:52 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1744 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:53.464007 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:53.464924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:53.472351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:53.480602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:53.481107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:53.481861 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:53.482320 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:53.483659 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:53.661933 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:53.669339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:53.670171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:53.677052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:53.677542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:53.677979 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:53.678677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:53.679285 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-990609af-2682-4f92-bd7a-6edddd0e754b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.219465 Oct 12 02:31:53.679395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:53.679711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:53.680066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 209/401] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:53 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:53.684431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:53.837715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:53.847278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:53.848849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:53.850607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3c6a9adb-4c17-4fce-86d3-b232fd001327 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1744 microversion: 2.20 time: 0.189044 Oct 12 02:31:53.851765 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 193/402] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:53 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1744 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:54.704749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:54.706021 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:54.712203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:54.712993 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:54.713483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:54.714569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:54.715026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:54.715614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:54.853157 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:54.861412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b59bed6f-8318-4a13-bc26-58225a1f3d77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.161641 Oct 12 02:31:54.861989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 210/403] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:54 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:54.879269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:54.879928 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:54.886536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:54.887014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:54.887441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:54.888123 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:54.888564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:54.889022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:55.036731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:55.044731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:55.046215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:55.047354 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-dbcfc9c0-d501-48ef-82c6-aa104284ec93 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.171945 Oct 12 02:31:55.048010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 194/404] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:54 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:55.143358 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:55.143868 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:55.152191 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:55.158488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:55.159050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:55.159774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:55.160322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:55.165074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:55.197480 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a80c48fc-4ec7-4013-9ea3-2dc34fa9c294 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/ips" status: 200 len: 150 microversion: 2.1 time: 0.062599 Oct 12 02:31:55.217093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 211/405] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:31:55 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/ips => generated 150 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:31:55.885843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:55.886722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:55.906369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:55.908135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:55.909519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:55.913223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:55.917423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:55.919159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:56.077998 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:56.078620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:56.089897 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:56.090199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:56.090602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:56.091323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:56.091783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:56.092806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:56.124290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:56.135516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-195931b9-1941-438c-9068-97aab4ad969c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.253988 Oct 12 02:31:56.136721 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 195/406] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:55 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:56.268494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:56.283146 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:56.285023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:56.286611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b6023b9d-d0a6-4589-9639-9f19417120fa tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.215168 Oct 12 02:31:56.287645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 212/407] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:56 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:56.403481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-984fd392-fd61-41aa-84b7-6bd9506a846c req-b366d3d5-3a89-440c-aa16-ea60c1f365b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48eb4bd4-2680-4fc6-85c0-dae85fb740fa", "name": "network-vif-plugged", "status": "completed", "tag": "0a31a866-f72c-4f7d-822f-7d54a4e3afec"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:56.433516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-984fd392-fd61-41aa-84b7-6bd9506a846c req-b366d3d5-3a89-440c-aa16-ea60c1f365b4 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:56.434242 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-984fd392-fd61-41aa-84b7-6bd9506a846c req-b366d3d5-3a89-440c-aa16-ea60c1f365b4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:56.434812 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-984fd392-fd61-41aa-84b7-6bd9506a846c req-b366d3d5-3a89-440c-aa16-ea60c1f365b4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:56.460464 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-984fd392-fd61-41aa-84b7-6bd9506a846c req-b366d3d5-3a89-440c-aa16-ea60c1f365b4 service nova] Creating event network-vif-plugged:0a31a866-f72c-4f7d-822f-7d54a4e3afec for instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa on hv-cinder-84914 Oct 12 02:31:56.470397 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-984fd392-fd61-41aa-84b7-6bd9506a846c req-b366d3d5-3a89-440c-aa16-ea60c1f365b4 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071037 Oct 12 02:31:56.471273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 196/408] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:31:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:31:56.983587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "960f293c-f2d1-48a5-8f68-2c9c685f2c9a"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:31:56.988636 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:57.004965 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:57.005704 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:57.007800 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:57.013333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:57.014478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:57.015278 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:57.164338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:57.165254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:57.180973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:57.181469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:57.181918 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:57.182722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:57.186838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:57.188839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:57.509632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:57.524570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5dc0905c-f974-40da-8d66-e180e77daf02 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.366468 Oct 12 02:31:57.525913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 197/409] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:57 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:57.530965 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:57.531851 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:57.538723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:57.539515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:57.540266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:57.541229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:57.543275 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:57.543964 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:57.638298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:31:57.657295 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d7f2901e-ffc1-4466-8072-d2a19b35ecf5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.677953 Oct 12 02:31:57.666280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 213/410] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:31:56 2022] POST /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments => generated 194 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:31:57.819754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:57.830040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:57.833146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:57.837013 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5779dcd7-19a7-4ac2-be49-0b8b4c0ede9a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.309813 Oct 12 02:31:57.848861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 198/411] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:57 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:31:58.552293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:58.553236 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:58.567525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:58.569369 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:58.577521 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:58.578859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:58.579557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:58.580224 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:58.849276 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:58.858292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-01e67b57-852a-4861-9e1e-60b6cb7377cc tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.310220 Oct 12 02:31:58.868360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:58.869577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:58.872006 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 214/412] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:58 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:31:58.882432 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:58.883205 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:58.883781 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:58.884875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:58.885468 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:58.886283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:59.186926 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:59.199152 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:59.202654 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:31:59.205036 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e678ca1c-3d18-42fb-bbaa-15a92a72842b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.340995 Oct 12 02:31:59.207180 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 199/413] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:31:58 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:31:59.394681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:59.395183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:59.401429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:59.402210 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:59.402960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:59.405842 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:59.406456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:59.407813 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:59.600291 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lazy-loading 'services' on Instance uuid 921f3627-5480-46cb-9810-fa6fcfde17ca {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:31:59.793161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d7f6d523-33bb-4614-8c4d-87b7fb726919 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "DELETE /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca/os-volume_attachments/2e4a5877-84d9-41fa-ace8-3cd3917d84e5" status: 202 len: 0 microversion: 2.1 time: 0.408652 Oct 12 02:31:59.798611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 215/414] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:31:59 2022] DELETE /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca/os-volume_attachments/2e4a5877-84d9-41fa-ace8-3cd3917d84e5 => generated 0 bytes in 415 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:31:59.897278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:31:59.898664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:31:59.906874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:59.907834 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:59.908512 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:31:59.909405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:31:59.909914 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:31:59.910565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:00.114229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:00.122766 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2b14edb1-7aa9-4fa1-8af9-a58439a3851e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.229545 Oct 12 02:32:00.123474 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 200/415] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:31:59 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:00.240359 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:00.241086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:00.247438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:00.247927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:00.248965 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:00.249933 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:00.250600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:00.251230 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:00.404652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:00.416835 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:00.419774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:00.421725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c1b7f9a9-550e-4638-b5e1-757e081705c1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.186116 Oct 12 02:32:00.423546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 216/416] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:00 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:01.150105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:01.151334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:01.159113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:01.166797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:01.167605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:01.168574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:01.169260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:01.170400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:01.380867 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:01.392642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-41d1ae6c-f95d-4512-a11b-eb9942fef939 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.246156 Oct 12 02:32:01.394381 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 201/417] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:01 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:01.457469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:01.458981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:01.471346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:01.472195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:01.473051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:01.477189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:01.477966 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:01.478622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:01.659299 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:01.669014 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:01.671775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:01.673095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-dadf7e69-57d7-44f4-8503-1116d9a96f85 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.219634 Oct 12 02:32:01.674867 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 217/418] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:01 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:02.416980 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:02.418315 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:02.428873 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:02.431855 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:02.433072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:02.434058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:02.434928 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:02.436359 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:02.610871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:02.620973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d2018c48-d1f9-486e-be05-e684917d118f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.208376 Oct 12 02:32:02.622337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 202/419] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:02 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:02.699444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:02.700075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:02.707913 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:02.708784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:02.709569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:02.710458 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:02.711065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:02.712254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:03.083563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:03.083563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:03.083787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:03.083787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a467a6bd-b1fa-4158-a8b0-64e74eec058b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.368352 Oct 12 02:32:03.083787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 218/420] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:02 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:03.643457 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:03.644069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:03.650096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:03.650802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:03.651409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:03.652272 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:03.653002 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:03.653682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:03.938372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:03.954668 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a18e3115-c161-4305-9437-e41a36f4b994 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1503 microversion: 2.1 time: 0.314944 Oct 12 02:32:03.955252 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 203/421] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:03 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1503 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:03.983403 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9ddf407a-0caa-4066-a60d-de5bb39acb0a"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:03.987280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:03.996478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:03.997517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:03.998412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:03.999499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:04.000243 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:04.001371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:04.030309 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] HTTP exception thrown: Cannot 'attach_volume' instance 3d9e5223-f388-4f16-b15a-73c59e86de18 while it is in vm_state rescued Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Request method failure captured: Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: POST /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18/os-volume_attachments HTTP/1.1 Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 74 Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaPYwNdGPkaZC6cHEUGZ0SkhncrUI5nVnVZxOFbWJ3oJ3B77igC0roWc-nMKs5IUYGUtOa_YaPxwFn9fOjAYUswTJuudkNK6KicjYPQGN35ZLr1BK_R3Kt1jRPqGECcut8i4CjPKiklVRegxUQM3z5ZDQ5sJ8YSuV1B4cub9phJoCc Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:32:04.032302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bd41b04d606949edbc9a3a2f88f991b2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: bcd7f1121dab4e8db4c747c51ed49240 Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {"volumeAttachment": {"volumeId": "9ddf407a-0caa-4066-a60d-de5bb39acb0a"}} Oct 12 02:32:04.033916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Cannot 'attach_volume' instance 3d9e5223-f388-4f16-b15a-73c59e86de18 while it is in vm_state rescued Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'server_id': '3d9e5223-f388-4f16-b15a-73c59e86de18', 'body': {'volumeAttachment': {'volumeId': '9ddf407a-0caa-4066-a60d-de5bb39acb0a'}}} Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3d9e5223-f388-4f16-b15a-73c59e86de18 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:32:04.034864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 3d9e5223-f388-4f16-b15a-73c59e86de18 while it is in vm_state rescued Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:32:04.036228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:32:04.037470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:32:04.037470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:32:04.037470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 3d9e5223-f388-4f16-b15a-73c59e86de18 while it is in vm_state rescued Oct 12 02:32:04.037470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:32:04.037470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Returning 409 to user: Cannot 'attach_volume' instance 3d9e5223-f388-4f16-b15a-73c59e86de18 while it is in vm_state rescued {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:32:04.038202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-709c205f-6313-45d0-ac2b-72b558d3381c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.060701 Oct 12 02:32:04.039188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 219/422] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:32:03 2022] POST /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18/os-volume_attachments => generated 152 bytes in 62 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 12 02:32:04.047951 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:04.048565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:04.056154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:04.056640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:04.057124 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:04.057821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:04.058501 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:04.058925 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:04.098559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:04.099342 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:04.109187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:04.110711 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:04.111486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:04.113323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:04.113999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:04.115185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:04.199450 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2909b86e-b22b-400a-b78b-b78f18f40f0a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18/action" status: 202 len: 0 microversion: 2.1 time: 0.155666 Oct 12 02:32:04.207729 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 204/423] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:32:04 2022] POST /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18/action => generated 0 bytes in 162 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:32:04.229749 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:04.230997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:04.250224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:04.251947 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:04.253955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:04.255752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:04.256566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:04.257797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:04.493166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:04.502167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9f4a85d3-8b73-407d-bf41-e7bcc72303ff tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1511 microversion: 2.1 time: 0.292855 Oct 12 02:32:04.506879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 205/424] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:04 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1511 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:04.537639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:04.553909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:04.557525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:04.559000 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-10412145-d44b-4108-bb79-24bc247a9f24 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.464538 Oct 12 02:32:04.562670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 220/425] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:04 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:32:05.529290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:05.530037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:05.538859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:05.539261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:05.542689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:05.546203 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:05.546884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:05.552398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:05.590114 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:05.590718 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:05.604104 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:05.604600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:05.605063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:05.625184 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:05.625781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:05.626229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:05.953467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:05.956987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:05.972545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-498165b7-6e79-4a85-be42-1a5cd7c41629 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1511 microversion: 2.1 time: 0.448082 Oct 12 02:32:06.006653 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:06.007007 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 206/426] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:05 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1511 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:06.017227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:06.019752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-06f1ed13-351c-401c-b75a-88b5aa2d3a4b tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.434004 Oct 12 02:32:06.023494 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-24cf4e7d-8021-4d18-bac4-acde5c5d7f25 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: .version_select at 0x7f59d37a63a0>, body: {"keypair": {"name": "tempest-keypair-61575296", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOpgBGpGHIyeQLxsppJjSMkUwcUWc3MFVwUEiJnak/mk7upwVs3gopyR7GvA0+sv21gGSgl0zcHgKQD7V59l1F5LEqOpzAMRCFpFfMy5cgm54JGG3nX0SdC4elORvY1keA=="}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:06.024441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 221/427] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:05 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:06.078092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-24cf4e7d-8021-4d18-bac4-acde5c5d7f25 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:32:06.087342 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-24cf4e7d-8021-4d18-bac4-acde5c5d7f25 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:32:06.181308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-24cf4e7d-8021-4d18-bac4-acde5c5d7f25 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.169689 Oct 12 02:32:06.182628 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 207/428] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:32:06 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:06.477125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:06.478147 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:06.486066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:06.486793 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:06.487840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:06.489398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:06.491088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:06.491809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:06.529890 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:32:06.667055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9fda2109-146c-481f-9fd1-71506b91dea1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "DELETE /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 204 len: 0 microversion: 2.1 time: 0.192948 Oct 12 02:32:06.668618 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 222/429] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:32:06 2022] DELETE /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:32:06.679824 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:06.681058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:06.695655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:06.698683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:06.699570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:06.703922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:06.704624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:06.706081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:07.018868 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:07.019962 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:07.028107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:07.029789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:07.030647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:07.032028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:07.033015 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:07.033735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:07.127953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:07.144737 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-53d624ec-d856-45c7-8423-b2822cdfcdee tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1513 microversion: 2.1 time: 0.468450 Oct 12 02:32:07.157448 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 208/430] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:06 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1513 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:07.174375 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:07.174994 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:07.197909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:07.202814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:07.203297 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:07.204035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:07.204504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:07.204965 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:07.272762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:07.288899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9fa44f3c-7ed7-41fe-a435-811a76da227a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1511 microversion: 2.1 time: 0.274029 Oct 12 02:32:07.289760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 223/431] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:07 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1511 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:07.498739 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:07.510713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:07.513147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:07.518610 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2799fb9a-3f50-4c1d-8d0e-abe37bdaaa04 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.350232 Oct 12 02:32:07.524888 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 209/432] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:07 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:08.177126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:08.177742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:08.187646 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:08.188754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:08.189843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:08.191995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:08.193250 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:08.194398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:08.314291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:08.315534 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:08.323256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:08.324136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:08.324871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:08.326561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:08.327433 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:08.328013 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:08.354378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:08.595928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:08.608179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bd42ca6b-4ecc-4b59-b26c-ad7bf875b489 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1513 microversion: 2.1 time: 0.438456 Oct 12 02:32:08.611689 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 224/433] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:08 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1513 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:08.624351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1616292913", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "21fdf3f0-6b90-400a-b2fc-5e540c8c18e8"}], "security_groups": [{"name": "tempest-securitygroup--1504510116"}], "key_name": "tempest-keypair-61575296", "user_data": "}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:08.773326 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] validate_networks() for [('21fdf3f0-6b90-400a-b2fc-5e540c8c18e8', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:32:08.820259 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:08.832271 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4f3ba646-6773-4296-be9a-a101d0075281 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1511 microversion: 2.1 time: 0.522404 Oct 12 02:32:08.834048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 210/434] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:08 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1511 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:08.845215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:08.845829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:08.853648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:08.854536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:08.855168 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:08.859707 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:08.860564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:08.861166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:09.033523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:09.042696 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:09.045573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:09.046911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-412a9058-7c10-4220-b710-eff54706ee06 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.210847 Oct 12 02:32:09.061348 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 211/435] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:08 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:09.307486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:32:09.308088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:32:09.308455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:32:09.308954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:32:09.309372 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:32:09.321234 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:09.322096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:09.322648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:09.323774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:09.324233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:09.324855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:09.342773 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:32:09.346765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:32:09.359508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:32:09.362039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:32:09.412778 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4f9bbf3d-54d0-48af-a0ce-5802533f137c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.796768 Oct 12 02:32:09.421674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 225/436] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:32:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 801 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:32:09.426661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e9b5714f-3a7e-419e-8f35-422061f170a4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:09.428078 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e9b5714f-3a7e-419e-8f35-422061f170a4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:09.496861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e9b5714f-3a7e-419e-8f35-422061f170a4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:09.504422 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e9b5714f-3a7e-419e-8f35-422061f170a4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1240 microversion: 2.1 time: 0.081706 Oct 12 02:32:09.505021 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 212/437] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:09 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1240 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:09.640036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:09.641032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:09.649593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:09.650416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:09.651061 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:09.652109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:09.652680 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:09.653345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:09.836404 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:09.850525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-caba1c64-92dc-4375-8d02-bee3c09cd094 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1513 microversion: 2.1 time: 0.212880 Oct 12 02:32:09.851312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 226/438] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:09 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1513 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:09.864199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:09.865364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:09.873868 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:09.874578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:09.875417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:09.876333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:09.877273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:09.877915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:10.053970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:10.062838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2c7130c4-d685-4805-9d68-d3f902b0929e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1511 microversion: 2.1 time: 0.203390 Oct 12 02:32:10.063401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 213/439] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:09 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1511 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:10.084261 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:10.084895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:10.091823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:10.092663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:10.093335 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:10.094034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:10.094486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:10.094906 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:10.295675 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:10.304575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:10.306064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:10.307211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-09cb01b7-26aa-4e97-9c58-6c21fbec9d8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1739 microversion: 2.20 time: 0.226912 Oct 12 02:32:10.308006 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 227/440] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:10 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1739 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:10.527954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:10.528620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:10.536591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:10.537426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:10.537902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:10.539130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:10.539787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:10.540540 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:10.625186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:10.635109 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-82fbc48d-0773-4227-aef8-fb19cccf5664 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1244 microversion: 2.1 time: 0.111593 Oct 12 02:32:10.635840 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 214/441] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:10 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1244 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:10.875172 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:10.875909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:10.885350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:10.885776 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:10.893067 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:10.894027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:10.894745 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:10.897895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.081646 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:11.089963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c587dc0d-d6dc-44bb-a844-3bc3c71fb4bf tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1513 microversion: 2.1 time: 0.218374 Oct 12 02:32:11.090637 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:11.092194 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:11.105762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 228/442] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:10 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1513 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:11.106797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:11.107543 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:11.108173 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.109864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:11.121313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:11.122072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.333992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:11.334897 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:11.345882 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:11.347189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:11.348008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.350480 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:11.351420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:11.352540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.353144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:11.361077 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:11.365930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-05cf4abd-fc09-429b-8da8-b38fa0adb661 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1517 microversion: 2.1 time: 0.282398 Oct 12 02:32:11.366510 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 215/443] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:11 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1517 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:11.612977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-160a7063-9a53-494f-bd35-57d5f4ae479a req-46164a01-52d0-4c9f-8996-ac13c03e8168 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "921f3627-5480-46cb-9810-fa6fcfde17ca", "name": "network-vif-deleted", "tag": "57cbb2f7-a82b-48d3-8cf3-b111c66a8260"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:11.638003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-160a7063-9a53-494f-bd35-57d5f4ae479a req-46164a01-52d0-4c9f-8996-ac13c03e8168 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:11.641979 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-160a7063-9a53-494f-bd35-57d5f4ae479a req-46164a01-52d0-4c9f-8996-ac13c03e8168 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:11.644481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-160a7063-9a53-494f-bd35-57d5f4ae479a req-46164a01-52d0-4c9f-8996-ac13c03e8168 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.678639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-160a7063-9a53-494f-bd35-57d5f4ae479a req-46164a01-52d0-4c9f-8996-ac13c03e8168 service nova] Creating event network-vif-deleted:57cbb2f7-a82b-48d3-8cf3-b111c66a8260 for instance 921f3627-5480-46cb-9810-fa6fcfde17ca on hv-cinder-84914 Oct 12 02:32:11.691770 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-160a7063-9a53-494f-bd35-57d5f4ae479a req-46164a01-52d0-4c9f-8996-ac13c03e8168 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084319 Oct 12 02:32:11.693694 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 216/444] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:11.705355 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:11.710410 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:11.724788 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:11.725294 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:11.725787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.726416 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:11.728049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:11.729014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:11.829704 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:11.838926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:11.840360 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:11.842098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-419e9b8c-3b64-432b-8744-156ed6f499d4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1721 microversion: 2.20 time: 0.511802 Oct 12 02:32:11.843867 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 229/445] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:11 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1721 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:11.858407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:11.868469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2bad9517-aa3d-45f5-97e1-da96e86e835e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1292 microversion: 2.1 time: 0.171224 Oct 12 02:32:11.869394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 217/446] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:11 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1292 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:12.116413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:12.117052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:12.135291 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:12.135801 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:12.136205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:12.136937 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:12.137890 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:12.139090 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:12.174841 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:12.175628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:12.176411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:12.177068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lazy-loading 'fault' on Instance uuid 921f3627-5480-46cb-9810-fa6fcfde17ca {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:32:12.297836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:12.306786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e2e8c632-e2f4-47e9-affa-ccd41fdfafa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1309 microversion: 2.1 time: 0.195672 Oct 12 02:32:12.308164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 230/447] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:12 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1309 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:12.885164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:12.886338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:12.889390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:12.895157 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:12.896029 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:12.899997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:12.901487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=82,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:12.906643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:12.907099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:12.907839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:12.908293 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:12.908727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:12.909581 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:12.910290 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:12.911530 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:12.913794 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:12.914704 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:12.915321 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:13.105165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:13.114376 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9ad6ffd0-2a9e-4cde-bcb0-42f4c77bdc68 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.225178 Oct 12 02:32:13.115623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 231/448] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:12 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:13.134072 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-88c65be7-f9f5-47d7-9e71-906d8c8b9bc5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'create', calling method: .version_select at 0x7f59d36a3dc0>, body: {"keypair": {"name": "tempest-keypair-1012864248", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGKyW0K8o+w7FewDR11pnqnmmj/ZleguHrAbA0Mg+9+dMUck3CdwhtqnkQTzBCgsU7Hbz2mglDheLC6lRoxgz8nsVthi+IJ5olsFR62BP4ojLAybrGu7rZNIJXX6lnrqCA=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:13.156617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-88c65be7-f9f5-47d7-9e71-906d8c8b9bc5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:32:13.160981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-88c65be7-f9f5-47d7-9e71-906d8c8b9bc5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for user bcd7f1121dab4e8db4c747c51ed49240 and project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:32:13.188500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-88c65be7-f9f5-47d7-9e71-906d8c8b9bc5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.071539 Oct 12 02:32:13.189265 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 232/449] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:32:13 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:13.244501 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:13.253953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:13.255689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:13.256895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1821618b-164a-4c96-9794-dd6d2df1bb9d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1721 microversion: 2.20 time: 0.390181 Oct 12 02:32:13.257497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 218/450] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:12 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1721 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:13.331894 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:13.333765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:13.340891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:13.342176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:13.342586 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:13.343462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:13.344083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:13.344665 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:13.383872 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:13.384597 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:13.385068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:13.385518 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lazy-loading 'fault' on Instance uuid 921f3627-5480-46cb-9810-fa6fcfde17ca {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:32:13.428545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:13.436429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9a7226ba-52f3-4517-b130-dbceee44dc2f tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1309 microversion: 2.1 time: 0.109594 Oct 12 02:32:13.437038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 233/451] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:13 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1309 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:14.141250 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:14.141998 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:14.148860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.149351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.149786 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.150646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.151270 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.151533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.284935 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:14.286366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:14.299437 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.300264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.301800 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.302553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.303065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.303493 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.395915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:14.410500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1d494b48-993a-4870-9c5d-86b6c558dc43 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.275038 Oct 12 02:32:14.417335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 219/452] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:14 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:14.423674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-f4bf4699-b443-479a-9108-2571a786c56a req-802bf785-b841-44af-8ed6-404fc38c7809 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48eb4bd4-2680-4fc6-85c0-dae85fb740fa", "name": "network-vif-unplugged", "status": "completed", "tag": "0a31a866-f72c-4f7d-822f-7d54a4e3afec"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:14.438406 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f4bf4699-b443-479a-9108-2571a786c56a req-802bf785-b841-44af-8ed6-404fc38c7809 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.439199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f4bf4699-b443-479a-9108-2571a786c56a req-802bf785-b841-44af-8ed6-404fc38c7809 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.439973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f4bf4699-b443-479a-9108-2571a786c56a req-802bf785-b841-44af-8ed6-404fc38c7809 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.461034 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-f4bf4699-b443-479a-9108-2571a786c56a req-802bf785-b841-44af-8ed6-404fc38c7809 service nova] Creating event network-vif-unplugged:0a31a866-f72c-4f7d-822f-7d54a4e3afec for instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa on hv-cinder-84914 Oct 12 02:32:14.467133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-f4bf4699-b443-479a-9108-2571a786c56a req-802bf785-b841-44af-8ed6-404fc38c7809 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.047940 Oct 12 02:32:14.468107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 220/453] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:14.474744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:14.476600 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:14.483993 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.484486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.485122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.486009 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.486613 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.487184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.518130 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:14.526937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:14.527883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:14.528624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:14.529493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lazy-loading 'fault' on Instance uuid 921f3627-5480-46cb-9810-fa6fcfde17ca {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:32:14.536675 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:14.538333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:14.540464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a7c01ded-5210-4766-92fe-d470d2280ce8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1721 microversion: 2.20 time: 0.261700 Oct 12 02:32:14.541650 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 234/454] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:14 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1721 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:32:14.639673 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:14.652091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-394e076e-67d0-4399-a480-7e2c1a85c5a5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 200 len: 1309 microversion: 2.1 time: 0.182473 Oct 12 02:32:14.653047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 221/455] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:14 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 1309 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:14.866802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-595396986", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d2208204-5dac-4ba9-9cf8-255af28eb635"}], "security_groups": [{"name": "tempest-securitygroup--1462813880"}], "key_name": "tempest-keypair-1012864248", "user_data": "}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:14.997942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] validate_networks() for [('d2208204-5dac-4ba9-9cf8-255af28eb635', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:32:15.378282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-8fc409ae-c529-477e-a26c-84585f21209e req-dacdf5bc-e7f5-470d-9fff-b242abfe5b07 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7dc118ab-0194-4cbb-b335-caf4c25cca7e", "tag": "79216b85-6ffe-4aa2-b822-11695d819ed3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:15.407437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8fc409ae-c529-477e-a26c-84585f21209e req-dacdf5bc-e7f5-470d-9fff-b242abfe5b07 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.409683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8fc409ae-c529-477e-a26c-84585f21209e req-dacdf5bc-e7f5-470d-9fff-b242abfe5b07 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.411198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8fc409ae-c529-477e-a26c-84585f21209e req-dacdf5bc-e7f5-470d-9fff-b242abfe5b07 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.433106 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-8fc409ae-c529-477e-a26c-84585f21209e req-dacdf5bc-e7f5-470d-9fff-b242abfe5b07 service nova] Creating event network-changed:79216b85-6ffe-4aa2-b822-11695d819ed3 for instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e on hv-cinder-84914 Oct 12 02:32:15.448864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-8fc409ae-c529-477e-a26c-84585f21209e req-dacdf5bc-e7f5-470d-9fff-b242abfe5b07 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072465 Oct 12 02:32:15.448864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 222/456] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:15.448864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:15.449423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:15.456446 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.457211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.457744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.458569 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.459115 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.460225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.501142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:32:15.501571 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:32:15.502039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:32:15.502443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:32:15.502849 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:32:15.511497 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.511974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.512405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.513707 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.514158 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.514581 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.534605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:32:15.538612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Getting quotas for user bcd7f1121dab4e8db4c747c51ed49240 and project bd41b04d606949edbc9a3a2f88f991b2. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:32:15.553551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:32:15.555713 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:32:15.606730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4bd4a240-158c-4a62-bb2c-a60e383f7763 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.745944 Oct 12 02:32:15.614237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 235/457] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:32:14 2022] POST /compute/v2.1/servers => generated 409 bytes in 754 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:32:15.620233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:15.620875 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:15.628099 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.628587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.629060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.629765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.630219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.630645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.684521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:15.694521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bca9ecd1-2272-4784-86e4-5f030415f39c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.250670 Oct 12 02:32:15.696559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 223/458] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:15 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:15.701895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1a6c1fb8-bca2-4e45-a518-c2a8a91f6130 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:15.702979 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1a6c1fb8-bca2-4e45-a518-c2a8a91f6130 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:15.770000 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1a6c1fb8-bca2-4e45-a518-c2a8a91f6130 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:15.784784 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1a6c1fb8-bca2-4e45-a518-c2a8a91f6130 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1245 microversion: 2.1 time: 0.086411 Oct 12 02:32:15.800726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 224/459] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:15 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1245 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:15.800726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:15.800726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: 921f3627-5480-46cb-9810-fa6fcfde17ca] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:15.800726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.801333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.801333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.812432 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:15.815636 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.822282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.822839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.830254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:15.833504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:15.834744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ee33d596-5899-49da-a360-154c31d98d67 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1664 microversion: 2.20 time: 0.219485 Oct 12 02:32:15.835639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 236/460] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:15 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1664 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:15.846779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] HTTP exception thrown: Instance 921f3627-5480-46cb-9810-fa6fcfde17ca could not be found. Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Request method failure captured: Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca HTTP/1.1 Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRibAWv4KbWVc6HhsH9Vu4uASpvDQ2d8uwxitHHc61pAl5Z0K_Iq5LjdO1RWnXS9a92ffwo8-SE9WT3KRO52id2ZHDNXujgbasBq52COoc50xCZeJmm9noLC6-V3TEpp-EMGvYZoXyN8UzwCLzTdWD1NgoQhANTntnAuZC1b0Wy-Q98A Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 81144c673e7b44dcb3b0f700ee628719 Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1936448719 Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:32:15.849743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/81144c673e7b44dcb3b0f700ee628719"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/81144c673e7b44dcb3b0f700ee628719"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/81144c673e7b44dcb3b0f700ee628719"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_81144c673e7b44dcb3b0f700ee628719", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-VolumesSnapshotTestJSON-1936448719 Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 81144c673e7b44dcb3b0f700ee628719 Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1936448719 Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-VolumesSnapshotTestJSON-1936448719-project Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: bbcc7dcbf32b4048879bbc77e0998b52 Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-VolumesSnapshotTestJSON-1936448719-project Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 921f3627-5480-46cb-9810-fa6fcfde17ca could not be found. Oct 12 02:32:15.851215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '921f3627-5480-46cb-9810-fa6fcfde17ca'} Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 921f3627-5480-46cb-9810-fa6fcfde17ca could not be found. Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:32:15.851997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 921f3627-5480-46cb-9810-fa6fcfde17ca could not be found. Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 921f3627-5480-46cb-9810-fa6fcfde17ca could not be found. Oct 12 02:32:15.853127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:32:15.854190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Returning 404 to user: Instance 921f3627-5480-46cb-9810-fa6fcfde17ca could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:32:15.854686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a8e44877-b250-470d-b182-a9dc827796f9 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca" status: 404 len: 111 microversion: 2.1 time: 0.067271 Oct 12 02:32:15.855249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 225/461] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:15 2022] GET /compute/v2.1/servers/921f3627-5480-46cb-9810-fa6fcfde17ca => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:32:15.864926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "02b78871-902d-4683-8a9a-c874f7431d50"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:15.881599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:15.894085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.894697 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.895215 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:15.900716 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:15.901809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:15.902407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.247360 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-379e3dc8-b923-4195-a7fa-22f7a639b3c2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.386866 Oct 12 02:32:16.248502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 237/462] 10.222.0.23 () {62 vars in 1364 bytes} [Wed Oct 12 02:32:15 2022] POST /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments => generated 188 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:32:16.380157 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:16.380772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:16.387081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.387578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.387985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.388667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.389136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.389552 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.433423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118865) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 12 02:32:16.503817 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c1a424a6-5982-458d-9c4c-7778c40fa71a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/action" status: 202 len: 0 microversion: 2.20 time: 0.127805 Oct 12 02:32:16.504577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 226/463] 10.222.0.23 () {62 vars in 1319 bytes} [Wed Oct 12 02:32:16 2022] POST /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/action => generated 0 bytes in 129 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:32:16.512509 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:16.513681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:16.524547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.525236 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.525844 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.526690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.527297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.531295 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.694827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:16.704387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:16.705873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:16.708139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b7b5c483-60ec-452e-b405-0bc79dad14da tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1750 microversion: 2.20 time: 0.198428 Oct 12 02:32:16.708841 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 238/464] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:16 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1750 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:16.722166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:16.723104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:16.729886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.730374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.730807 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.731531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.732016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.732437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.813725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:16.814464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:16.823086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.823758 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.824354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.825130 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:16.826284 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:16.826735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:16.899661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:16.910331 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-511c7ca8-f41f-477b-b830-70013a777b0d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1249 microversion: 2.1 time: 0.101669 Oct 12 02:32:16.911119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 239/465] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:16 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1249 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:16.911238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:16.932404 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a28c884f-c0ba-4a70-a4ee-c269244a3509 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.216563 Oct 12 02:32:16.933328 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 227/466] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:16 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:17.737690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:17.738286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:17.744819 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:17.745307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:17.745744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:17.746437 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:17.746928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:17.747929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:17.883549 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:17.891938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:17.893395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:17.894527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-72d7a3d4-d39c-45e9-9947-9d3bc8c4eefd tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1750 microversion: 2.20 time: 0.160995 Oct 12 02:32:17.895071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 240/467] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:17 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1750 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:17.933154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:17.933742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:17.939605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:17.940111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:17.940541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:17.941248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:17.941701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:17.942123 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:17.955770 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:17.956384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:17.962347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:17.962812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:17.963278 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:17.963994 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:17.964454 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:17.964930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:18.002801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:18.011436 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-87cdfcb3-fd05-4f09-bad5-c70b54728bd6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1249 microversion: 2.1 time: 0.082092 Oct 12 02:32:18.012012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 228/468] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:17 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1249 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:18.103226 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:18.111650 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-488bd5ca-75b6-4327-92b7-8f5571523b36 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.159918 Oct 12 02:32:18.112465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 241/469] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:17 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:18.919624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:18.920237 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:18.926276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:18.926758 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:18.927193 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:18.927880 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:18.928329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:18.928788 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:19.036407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:19.037904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:19.047148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:19.047994 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:19.048625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:19.049551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:19.050015 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:19.050441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:19.072626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:19.083323 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:19.083904 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:19.085320 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c66eda2a-21e5-4548-bf57-7352d4f09568 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1750 microversion: 2.20 time: 0.169541 Oct 12 02:32:19.085938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 229/470] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:18 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1750 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:19.116229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:19.124661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-737baf73-9008-4496-84fb-655a53216f47 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1249 microversion: 2.1 time: 0.093015 Oct 12 02:32:19.125408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 242/471] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:19 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1249 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:19.134219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:19.135202 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:19.141805 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:19.142284 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:19.142712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:19.143643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:19.144101 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:19.144515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:19.163432 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:19.280916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:19.288537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7eca99cc-625e-4137-a5ab-f77cb44b614a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.158223 Oct 12 02:32:19.289169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 230/472] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:19 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:20.112772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:20.114127 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:20.124754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:20.125633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:20.126218 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:20.127083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:20.127689 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:20.128271 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:20.152548 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:20.153821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:20.164117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:20.164937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:20.165636 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:20.166497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:20.169945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:20.170691 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:20.344987 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:20.358861 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:20.360936 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:20.362631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-60b76e4a-3e4a-41a4-8817-44694c05de8a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1752 microversion: 2.20 time: 0.254255 Oct 12 02:32:20.373167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 243/473] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:20 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1752 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:20.387561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:20.389210 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:20.396849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:20.404870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7ec62869-e7f1-4ce9-a9e7-7af4501bd9d5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1297 microversion: 2.1 time: 0.260096 Oct 12 02:32:20.405428 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 231/474] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:20 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1297 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:20.418613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:20.419888 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:20.420386 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:20.421242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:20.421764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:20.422260 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:20.426620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:20.565776 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:20.574042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6486f8ff-3fb2-464f-a0f5-a6c22bbab7e3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.199547 Oct 12 02:32:20.574624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 244/475] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:20 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:21.393814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:21.394557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:21.400589 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:21.401110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:21.401541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:21.402722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:21.403178 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:21.403602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:21.431930 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:21.435949 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:21.456393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:21.457167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:21.457764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:21.458963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:21.467764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:21.468682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:21.555768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:21.565362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:21.566809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:21.567556 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:21.567969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-617d4b2e-6028-4266-8bda-9a1855a6eb4f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.179381 Oct 12 02:32:21.568735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 232/476] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:21 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:21.576059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-68dd6917-eefc-460c-a49c-00568278c449 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1303 microversion: 2.1 time: 0.150696 Oct 12 02:32:21.576836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 245/477] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:21 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1303 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:21.600977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:21.601586 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:21.607930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:21.608407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:21.608882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:21.609745 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:21.610197 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:21.610701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:21.782401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:21.796495 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-589014fc-bcdb-4d12-a76d-29248bde747b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.199321 Oct 12 02:32:21.797446 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 233/478] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:21 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:22.595390 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:22.596723 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:22.600430 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:22.601287 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:22.604219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:22.604730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:22.605321 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:22.605855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:22.606503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:22.606866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:22.608541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:22.609542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:22.609542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:22.616402 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:22.616924 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:22.617360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:22.783827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:22.793137 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0ffca104-b760-43db-a2b6-cb1f05dedd6d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.196344 Oct 12 02:32:22.794402 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 234/479] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:22 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:22.820277 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:22.821127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:22.827437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:22.828183 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:22.828816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:22.829670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:22.830280 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:22.830844 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:22.858337 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:22.873507 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:22.875629 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:22.876984 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ac4ed089-ec61-4dff-8d62-336bb81ff398 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.286150 Oct 12 02:32:22.877977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 246/480] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:22 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:22.899283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-1bb6c14a-942a-4de5-87c0-24a3a76cb138 req-eae44653-a134-4964-b00c-009426bbeb94 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac5527bc-4e21-4157-9415-41e66105dc9c", "tag": "3c705692-0c55-4ed6-88ed-b7ba46d3442f"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:22.909622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-1bb6c14a-942a-4de5-87c0-24a3a76cb138 req-eae44653-a134-4964-b00c-009426bbeb94 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:22.910233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-1bb6c14a-942a-4de5-87c0-24a3a76cb138 req-eae44653-a134-4964-b00c-009426bbeb94 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:22.910731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-1bb6c14a-942a-4de5-87c0-24a3a76cb138 req-eae44653-a134-4964-b00c-009426bbeb94 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:22.933469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-1bb6c14a-942a-4de5-87c0-24a3a76cb138 req-eae44653-a134-4964-b00c-009426bbeb94 service nova] Creating event network-changed:3c705692-0c55-4ed6-88ed-b7ba46d3442f for instance ac5527bc-4e21-4157-9415-41e66105dc9c on hv-cinder-84914 Oct 12 02:32:22.940291 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-1bb6c14a-942a-4de5-87c0-24a3a76cb138 req-eae44653-a134-4964-b00c-009426bbeb94 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045587 Oct 12 02:32:22.941069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 247/481] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:23.006120 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:23.014232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c5a8e199-c741-479d-b8d5-0ed710be5e0a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.197640 Oct 12 02:32:23.028794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 235/482] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:22 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:23.820556 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:23.821593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:23.828726 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:23.829312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:23.829747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:23.830474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:23.830923 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:23.831337 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:23.902734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:23.903367 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:23.909377 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:23.909863 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:23.910301 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:23.910990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:23.911607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:23.912173 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:23.992534 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:24.001805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-99f73505-4202-42c7-8071-f84b3084074f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.185970 Oct 12 02:32:24.002485 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 248/483] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:23 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:24.039533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:24.040146 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:24.048345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:24.048893 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:24.049332 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:24.050033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:24.050489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:24.050910 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:24.135827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:24.146807 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:24.148558 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:24.149747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d3f06e07-ea02-4612-b5fa-50495347a432 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.251023 Oct 12 02:32:24.150460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 236/484] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:23 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:24.187008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:24.195111 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0c595d7d-f683-4a6a-87ff-cd7eb12b6a9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.159519 Oct 12 02:32:24.195702 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 249/485] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:24 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:25.025255 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:25.025851 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:25.032396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:25.032913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:25.033349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:25.034027 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:25.034476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:25.034891 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:25.177576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:25.178190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:25.185325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:25.185872 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:25.186307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:25.187268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:25.187964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:25.188388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:25.224058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:25.232462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-dfafd4e4-ed00-43fa-8c34-7884dd9f5109 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.210929 Oct 12 02:32:25.233674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 237/486] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:25 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:25.238759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:25.239588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:25.247185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:25.248016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:25.249046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:25.249739 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:25.250375 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:25.250941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:25.485957 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:25.494796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-021c00c6-b088-448a-a761-f3ba507a930d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.259941 Oct 12 02:32:25.495526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 238/487] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:25 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:25.511823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:25.527527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:25.529537 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:25.531483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0e24f786-d38d-4d32-a8e0-1046e5c0e097 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.359467 Oct 12 02:32:25.532048 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 250/488] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:25 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:26.258524 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:26.260329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:26.266636 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=8 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:26.269384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:26.270105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:26.270970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:26.271690 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:26.272305 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:26.272944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:26.327269 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-131818311", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "networks": [{"uuid": "7c8feeb4-fea7-46ca-9f80-d14f4cc37df8"}]}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:26.358199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=4 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:26.359661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=16 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:26.454771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:26.462948 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c9e4e5be-0c19-48e8-9397-4bb2d02dbc98 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.208066 Oct 12 02:32:26.463624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 239/489] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:26 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:26.519199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:26.519797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:26.525971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:26.526454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:26.526876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:26.527558 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:26.528061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:26.529030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:26.682380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:26.690533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ab6b35c8-df55-4384-b551-7b6f585ea221 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.175128 Oct 12 02:32:26.691107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 240/490] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:26 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:26.695561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:26.696162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:26.706661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:26.707364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:26.708091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:26.709028 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:26.709478 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:26.709899 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:26.854823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:26.865265 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:26.867697 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:26.869351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d3b50631-d28e-41d0-a225-3b1ff490092f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.177274 Oct 12 02:32:26.870085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 241/491] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:26 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:27.287699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-e8390f95-4eb7-4e27-8b8b-d96c6bb40b9d req-13d87290-793b-46cb-a534-c5d548ee1de2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dc118ab-0194-4cbb-b335-caf4c25cca7e", "name": "network-vif-plugged", "status": "completed", "tag": "79216b85-6ffe-4aa2-b822-11695d819ed3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:27.292749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] validate_networks() for [('7c8feeb4-fea7-46ca-9f80-d14f4cc37df8', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:32:27.299055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-e8390f95-4eb7-4e27-8b8b-d96c6bb40b9d req-13d87290-793b-46cb-a534-c5d548ee1de2 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.299541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-e8390f95-4eb7-4e27-8b8b-d96c6bb40b9d req-13d87290-793b-46cb-a534-c5d548ee1de2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.299967 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-e8390f95-4eb7-4e27-8b8b-d96c6bb40b9d req-13d87290-793b-46cb-a534-c5d548ee1de2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.318658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-e8390f95-4eb7-4e27-8b8b-d96c6bb40b9d req-13d87290-793b-46cb-a534-c5d548ee1de2 service nova] Creating event network-vif-plugged:79216b85-6ffe-4aa2-b822-11695d819ed3 for instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e on hv-cinder-84914 Oct 12 02:32:27.325429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-e8390f95-4eb7-4e27-8b8b-d96c6bb40b9d req-13d87290-793b-46cb-a534-c5d548ee1de2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041206 Oct 12 02:32:27.326401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 242/492] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:32:27.488394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:27.489338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:27.497713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.498274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.502549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.508748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.509609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.510070 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.715902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:27.725522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ec10b140-953c-44bd-b7b0-011dfc413c40 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.241718 Oct 12 02:32:27.740909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 243/493] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:27 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:27.740909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:27.740909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:27.740909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.741394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.741394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.741394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.741644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.742379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.773682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:32:27.773682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:32:27.774316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:32:27.779612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:32:27.782038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:32:27.803970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.806258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.806772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.808495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.808998 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.809449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.834804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Getting quotas for project 81144c673e7b44dcb3b0f700ee628719. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:32:27.841301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Getting quotas for user bbcc7dcbf32b4048879bbc77e0998b52 and project 81144c673e7b44dcb3b0f700ee628719. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:32:27.871545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:32:27.874388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:32:27.942674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cad8f7f1-393f-4747-a75f-87cc5782d1ca tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.619251 Oct 12 02:32:27.946277 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 251/494] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:32:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 1623 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:32:27.958184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:27.958926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:27.959638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:27.966564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-746da775-3b9e-4cb2-a4c2-4d3e82041ee7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.239172 Oct 12 02:32:27.976748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 244/495] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:27 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:27.978916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0af28c89-209e-491d-8a73-d92a0bc00074 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:27.979527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0af28c89-209e-491d-8a73-d92a0bc00074 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:27.983365 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.984048 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.984566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:27.985388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:27.985785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:27.986202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:28.047809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0af28c89-209e-491d-8a73-d92a0bc00074 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:28.063772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0af28c89-209e-491d-8a73-d92a0bc00074 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1218 microversion: 2.1 time: 0.086924 Oct 12 02:32:28.069716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 245/496] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:27 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1218 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:28.210946 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:28.221910 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:28.223516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:28.225327 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5799438c-b971-4a79-93a7-0bd0fa87915e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.276815 Oct 12 02:32:28.228839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 252/497] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:27 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:28.751942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:28.752546 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:28.758902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:28.759392 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:28.759824 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:28.760527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:28.761008 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:28.761431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:28.917877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:28.926284 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6402a7ac-e12a-4539-851a-173bd043736c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.178235 Oct 12 02:32:28.926859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 246/498] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:28 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:28.991808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:28.992354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:28.999303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:28.999774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.000198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:29.000898 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:29.001347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.001756 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:29.095111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:29.095769 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:29.102239 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:29.102971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.103612 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:29.104469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:29.105105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.105767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:29.198214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:29.204796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:29.210557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1560f0bd-c1e5-45e4-8c10-5bb22b7e19c7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.222640 Oct 12 02:32:29.211575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 253/499] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:28 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:29.214286 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ecaa78ef-90a3-4227-910a-04e1d684e57d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1222 microversion: 2.1 time: 0.124442 Oct 12 02:32:29.215128 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 247/500] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:29 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1222 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:29.253799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:29.254321 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:29.260981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:29.261475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.261909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:29.262975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:29.263433 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.263858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:29.405171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:29.414030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:29.415481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:29.416649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-40714fcf-7bbb-4d4a-8763-15eb570b6a33 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.168614 Oct 12 02:32:29.417229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 254/501] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:29 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:29.950475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:29.951124 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:29.957234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:29.957802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.958235 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:29.958932 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:29.959383 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:29.959835 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:30.107593 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:30.111248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:30.117116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9703b715-36d5-4612-a3a0-141301856f0e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.170989 Oct 12 02:32:30.117944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 248/502] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:29 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:30.238036 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:30.238477 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:30.238963 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:30.239510 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:30.244923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:30.245412 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:30.245850 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:30.246589 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:30.247558 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:30.247981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:30.248460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:30.248989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:30.249535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:30.250351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:30.250887 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:30.251368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:30.329214 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:30.346320 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0cf0e8e8-7142-4e77-8da6-b47701ba083a tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1222 microversion: 2.1 time: 0.112171 Oct 12 02:32:30.356473 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 249/503] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:30 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1222 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:30.414433 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:30.426937 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-18b3ed04-2c93-4aca-b608-cd40c414615a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.192700 Oct 12 02:32:30.428471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 255/504] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:30 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:30.442988 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:30.445675 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:30.452279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:30.452793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:30.453221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:30.453912 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:30.454358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:30.454774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:30.594375 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:30.603675 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:30.605337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:30.607138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-aad1d983-7d9a-485b-80c7-355d03bc88be tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.167593 Oct 12 02:32:30.607974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 250/505] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:30 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:31.143344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:31.144089 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:31.155107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.155593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.156024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.156722 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.157443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.157793 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.299845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:31.308677 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a55d93ab-9e27-4dc6-8c2a-2e66d49ed727 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.169233 Oct 12 02:32:31.310447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 256/506] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:31 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:31.378963 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:31.379534 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:31.385540 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.386017 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.386476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.387808 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.388256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.388671 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.450714 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:31.451316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:31.457179 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:31.457478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.457960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.458397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.459288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.459749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.460171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.465917 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-96358e0d-38b4-4d67-bdbf-d15de6e7c1a7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1222 microversion: 2.1 time: 0.089994 Oct 12 02:32:31.466827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 251/507] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:31 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1222 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:31.601560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:31.616405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-194c87ea-683f-4027-80df-175319dc8e4e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.163466 Oct 12 02:32:31.617309 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 257/508] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:31 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:31.633623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:31.634189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:31.640999 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.641485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.641918 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.642615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:31.643063 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:31.643572 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:31.782407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:31.791500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:31.793156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:31.794854 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9d662dd2-510e-4623-bbfe-511a53b25a81 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.166059 Oct 12 02:32:31.795408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 252/509] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:31 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:32.332412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:32.333202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:32.339179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.339848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.340885 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.341594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.342292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.342748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.495342 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:32.496634 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:32.502842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.503566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.504183 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.505091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.505488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:32.505892 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.506473 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.514731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-550aa1b8-f49c-47be-8ff6-b34e387e3ac2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.185270 Oct 12 02:32:32.515710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 258/510] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:32 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:32.572349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:32.581842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0043c991-a085-4bf1-9fad-2bf6511e3d93 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1222 microversion: 2.1 time: 0.095825 Oct 12 02:32:32.582752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 253/511] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:32 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1222 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:32.634749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:32.635508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:32.642217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.642705 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.643247 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.644060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.644520 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.645095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.781789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:32.789720 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-106a82ff-e91e-49e5-92b8-166e335bfa54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.158820 Oct 12 02:32:32.790521 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 259/512] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:32 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:32.819387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:32.820082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:32.826682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.827171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.827811 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.828519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:32.828995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:32.829603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:32.981685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:32.991653 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:32.993206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:32.994563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0e5c0ab2-899c-458d-86ae-815b8f5f8963 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.179113 Oct 12 02:32:32.995385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 254/513] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:32 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:33.538859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:33.539447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:33.542517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:33.553400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:33.553897 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:33.554334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:33.555038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:33.555501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:33.555928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:33.607390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:33.608662 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:33.615277 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:33.615805 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:33.616262 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:33.616982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:33.617443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:33.617869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:33.693535 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:33.703717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fb35ffff-d88e-4c63-b663-c50d47a7ac66 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1222 microversion: 2.1 time: 0.100071 Oct 12 02:32:33.704550 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 255/514] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:33 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1222 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:33.711864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:33.720167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0d6775b0-21c9-48e5-a194-e2eff46e52b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.185038 Oct 12 02:32:33.728749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 260/515] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:33 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:33.815984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:33.816521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:33.823914 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:33.825251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:33.825704 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:33.826396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:33.826842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:33.827291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:33.977243 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:33.987177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-104f56fc-99c9-46c1-b68d-a3519bfee0f9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.174773 Oct 12 02:32:33.988317 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 256/516] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:33 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:34.023908 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:34.024918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:34.031639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:34.032346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:34.033117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:34.033980 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:34.034582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:34.035161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:34.192237 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:34.201319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:34.202796 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:34.207914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-314629cb-a3cf-4732-bf86-ee08a6b6a16c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.185866 Oct 12 02:32:34.208511 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 261/517] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:34 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:34.727037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:34.727649 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:34.734061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:34.734545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:34.734986 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:34.735671 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:34.736129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:34.736581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:34.743369 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:34.743978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:34.751033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:34.751517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:34.751950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:34.752670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:34.753204 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:34.753632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:34.758460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:34.799507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:34.808564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-49006150-82d3-41f6-a331-6f610d65bad4 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1222 microversion: 2.1 time: 0.085469 Oct 12 02:32:34.809749 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 257/518] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:34 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1222 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:34.972547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:34.981896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-02d7cf32-c03b-4614-903e-77bfd97bbdc0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.242373 Oct 12 02:32:34.983955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 262/519] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:34 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:35.010767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:35.011475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:35.017254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:35.017725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:35.018148 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:35.018846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:35.019289 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:35.019699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:35.146314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:35.154470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-88c511de-7b65-4cf4-bdec-f3faa7c010f6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.147507 Oct 12 02:32:35.155029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 258/520] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:35 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:35.233863 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:35.234461 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:35.240515 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:35.241026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:35.241456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:35.242128 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:35.242577 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:35.242992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:35.372575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:35.381039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:35.382473 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:35.383620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4bca531a-f09f-4b8f-b1d1-8373c81ecad9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.154174 Oct 12 02:32:35.384171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 263/521] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:35 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:35.831398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:35.832282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:35.838204 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:35.838684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:35.839119 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:35.839799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:35.840843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:35.841264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:35.916358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:35.923851 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f613490e-9d04-4464-b75b-059e9849e414 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.096310 Oct 12 02:32:35.924414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 259/522] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:35 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:36.006582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:36.007214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:36.014951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.015444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.015879 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:36.016566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.017125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.017547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:36.148533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:36.156286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8cd641af-1a94-4e9d-b70a-0270fd4ec604 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.153475 Oct 12 02:32:36.156885 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 264/523] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:36 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:36.176493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:36.177251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:36.183099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.183577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.184495 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:36.185200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.185651 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.186077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:36.312987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:36.320505 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-48ace3e7-3687-48ce-8a80-46ca5d5a97a5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.147790 Oct 12 02:32:36.321089 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 260/524] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:36 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:36.408305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:36.408933 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:36.414850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.415324 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.415751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:36.416455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.416935 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.417355 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:36.545726 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:36.554176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:36.555614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:36.556781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e6f78bba-d5ca-4acf-9c61-67ad975d3168 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.152303 Oct 12 02:32:36.557338 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 265/525] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:36 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:36.947080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:36.947688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:36.954140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.954624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.955052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:36.955729 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:36.956177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:36.956595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:37.032024 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:37.039616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-73cc39d9-cc50-4ce0-b8bd-ae5a87b2b138 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.096269 Oct 12 02:32:37.040172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 261/526] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:36 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:37.180094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:37.180720 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:37.188031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:37.188594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:37.189116 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:37.189890 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:37.190431 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:37.190940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:37.327012 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:37.335839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-67048f2f-198d-4942-8536-4c9dfe2de955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.160264 Oct 12 02:32:37.336499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 266/527] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:37 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:37.343624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:37.344288 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:37.347388 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:37.351726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:37.352209 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:37.352632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:37.353346 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:37.353801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:37.354223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:37.490881 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:37.499039 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b26b1ea8-b8c5-441b-b1b7-c6bb63255aaf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.159822 Oct 12 02:32:37.499604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 262/528] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:37 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:37.582426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:37.583028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:37.588922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:37.589407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:37.589846 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:37.590572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:37.591071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:37.591544 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:37.719441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:37.728104 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:37.729665 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:37.731324 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-143ec61f-85ad-4116-a6d7-f8223b508c29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.152260 Oct 12 02:32:37.731877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 267/529] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:37 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:38.067558 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:38.068408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:38.072077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:38.081276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.082220 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.082882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.083811 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.084541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.085340 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.165408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:38.175878 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2cdbec0b-46f6-46f8-b831-0ae40ff29dbd tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.115880 Oct 12 02:32:38.176622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 263/530] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:38 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:38.359727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:38.360263 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:38.362393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:38.363068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=6 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:38.369164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.369652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.370091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.370782 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.371238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.371670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.512854 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:38.520469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f3748b64-9b37-4fb3-b23e-125b81d543f6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.165223 Oct 12 02:32:38.521065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 268/531] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:38 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:38.523383 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:38.524051 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:38.530601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.531081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.531511 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.532187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.532661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.533231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.668249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:38.676599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-39eeffc6-016e-4041-8e56-027114274af5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.157507 Oct 12 02:32:38.677189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 264/532] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:38 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:38.758880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:38.759487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:38.765895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.766381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.766813 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.767507 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:38.767961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:38.768386 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:38.903739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:38.912346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:38.913825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:38.914990 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3300936c-fa34-48d9-8bbb-3997eed5a42f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.160062 Oct 12 02:32:38.915550 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 269/533] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:38 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:39.198122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:39.198767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:39.205182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.205665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.206099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:39.206778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.207232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.207650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:39.225187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:39.269801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:39.277702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8af3f967-94f2-4699-984f-483db8c3819e tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.083344 Oct 12 02:32:39.278280 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 265/534] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:39 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:39.545156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:39.545683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:39.552100 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.552582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.553067 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:39.553752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.554201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.554618 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:39.684066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:39.691759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b2d4380a-69a3-4251-bb8d-627a3f31a790 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.150968 Oct 12 02:32:39.692326 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 270/535] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:39 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:39.700557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:39.701210 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:39.707877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.708499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.709066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:39.709839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.710335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.710845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:39.840460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:39.847972 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4a48e326-04f0-41ab-8325-32b8d9ec4fc2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.151270 Oct 12 02:32:39.848537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 266/536] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:39 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:39.940106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:39.940752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:39.946604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.947084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.947512 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:39.948190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:39.948638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:39.949076 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:40.079444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:40.091132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:40.093877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:40.096065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c9877944-2076-4e87-82cd-c057805e927d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.159622 Oct 12 02:32:40.097102 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 271/537] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:39 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:40.300527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:40.301177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:40.307467 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:40.307954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:40.308390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:40.309126 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:40.309578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:40.309983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:40.391116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:40.398784 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a1367d0f-3cf9-4df8-923a-ed9c95085e68 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.102226 Oct 12 02:32:40.399344 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 267/538] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:40 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:40.714742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:40.715354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:40.721813 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:40.722303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:40.722740 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:40.723427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:40.723886 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:40.724308 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:40.852284 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:40.860655 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0a477a48-ecb9-4a27-b5e0-e025a428edf5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.149683 Oct 12 02:32:40.861265 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 272/539] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:40 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:40.872227 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:40.873445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:40.880992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:40.881474 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:40.881900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:40.882585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:40.883031 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:40.883439 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:41.015813 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:41.024771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9b3ae670-a846-4d32-968b-3757be3de9cb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.157040 Oct 12 02:32:41.025445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 268/540] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:40 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:41.122543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:41.123151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:41.129080 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:41.129564 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:41.129997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:41.130679 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:41.131132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:41.131555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:41.275919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:41.288038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:41.289738 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:41.291367 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-603b7bde-422c-4b13-a116-8235fb796e9c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.173157 Oct 12 02:32:41.292024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 273/541] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:41 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:41.421745 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:41.422636 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:41.429621 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:41.430280 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:41.431952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:41.431952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:41.432544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:41.433113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:41.501607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:41.510409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-284a854a-ea92-4463-a5d7-c4e14553bbff tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.092898 Oct 12 02:32:41.511037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 269/542] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:41 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:41.883855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:41.884437 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:41.890965 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:41.891450 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:41.891876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:41.892595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:41.893068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:41.893482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:42.023002 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:42.031371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8cbbe150-5ab5-4f3b-8a21-b3bc6c9c8731 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.151317 Oct 12 02:32:42.031985 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 274/543] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:41 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:42.050053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:42.050665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:42.056605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:42.057121 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:42.057552 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:42.058238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:42.058761 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:42.059187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:42.193340 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:42.201275 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d9a8e2dc-3c88-4fc5-b093-77c51e0d07da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.155141 Oct 12 02:32:42.201849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 270/544] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:42 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:42.317580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:42.318296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:42.324875 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:42.325358 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:42.325787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:42.326474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:42.326923 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:42.327339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:42.473395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:42.483265 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:42.485004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:42.486476 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-75b204e4-8db1-4794-b2ec-d3a49573ca91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.173501 Oct 12 02:32:42.487334 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 275/545] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:42 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:42.534221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:42.534831 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:42.540785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:42.541230 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:42.541667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:42.542353 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:42.542806 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:42.543226 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:42.603216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:42.610864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a387f457-3e39-4037-bc62-371f3736ed5d tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.080560 Oct 12 02:32:42.611426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 271/546] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:42 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:43.055313 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:43.056017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:43.064265 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.064902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.065421 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.066296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.066812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.067312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.197390 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:43.205827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-609649d7-c9d3-4d5c-aa84-03f890fa390f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.154619 Oct 12 02:32:43.206505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 276/547] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:43 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:43.225375 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:43.226049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:43.233492 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.234073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.234567 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.235350 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.235861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.236318 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.374169 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-151ab4ea-170d-4594-b075-a6c687ae775f req-a99ad0eb-f260-4fdf-85e2-8b3ab12f9874 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac5527bc-4e21-4157-9415-41e66105dc9c", "name": "network-vif-plugged", "status": "completed", "tag": "3c705692-0c55-4ed6-88ed-b7ba46d3442f"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:43.377376 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:43.384107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-151ab4ea-170d-4594-b075-a6c687ae775f req-a99ad0eb-f260-4fdf-85e2-8b3ab12f9874 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.384671 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-151ab4ea-170d-4594-b075-a6c687ae775f req-a99ad0eb-f260-4fdf-85e2-8b3ab12f9874 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.385424 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-151ab4ea-170d-4594-b075-a6c687ae775f req-a99ad0eb-f260-4fdf-85e2-8b3ab12f9874 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.385543 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0f2823e3-3e9e-4afd-9cf1-985b5c4f28ce tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.164407 Oct 12 02:32:43.386366 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 272/548] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:43 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:43.398949 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-151ab4ea-170d-4594-b075-a6c687ae775f req-a99ad0eb-f260-4fdf-85e2-8b3ab12f9874 service nova] Creating event network-vif-plugged:3c705692-0c55-4ed6-88ed-b7ba46d3442f for instance ac5527bc-4e21-4157-9415-41e66105dc9c on hv-cinder-84914 Oct 12 02:32:43.404944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-151ab4ea-170d-4594-b075-a6c687ae775f req-a99ad0eb-f260-4fdf-85e2-8b3ab12f9874 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034954 Oct 12 02:32:43.405755 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 277/549] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:32:43.513802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:43.514409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:43.520443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.520973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.521620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.522096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.522551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.522978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.635289 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:43.636041 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:43.642385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.642875 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.643325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.644021 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:43.644511 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:43.644952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:43.664432 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:43.678195 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:43.681466 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:43.683522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ba2dfd17-6d97-448e-907c-8b858347c75e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.173828 Oct 12 02:32:43.684472 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 273/550] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:43 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:43.707450 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:43.715740 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7bbc6e93-5979-4f1e-ae92-6d3c40ed1d52 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.085026 Oct 12 02:32:43.716305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 278/551] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:43 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:44.230579 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:44.231138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:44.237288 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.237768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.238199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.238882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.239331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.239745 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.367417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:44.375465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-45aa6683-2684-4adf-ba76-9408a5de34cf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.148794 Oct 12 02:32:44.376027 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 274/552] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:44 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:44.407350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:44.407960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:44.414411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.414894 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.415326 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.416008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.416462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.416921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.434934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:44.555056 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:44.563254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c578c6ba-0d03-4cf4-bb4d-88939a9cdd50 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.159796 Oct 12 02:32:44.563849 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 279/553] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:44 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:44.709656 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:44.710256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:44.716295 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.716818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.717258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.717948 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.718406 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.718827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.739103 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:44.739710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:44.747064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.747780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.748214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.749025 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:44.749479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:44.749902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:44.825171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:44.834280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2c08d123-ba3e-4874-9bfc-5aade29f28f3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.098982 Oct 12 02:32:44.834881 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 280/554] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:44 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:44.864280 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:44.873810 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:44.875261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:44.876419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b7617852-e905-4c57-b2b3-cfc1666a528e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.170642 Oct 12 02:32:44.877005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 275/555] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:44 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:45.399045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:45.399652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:45.406348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.406834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.407281 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.407957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.408538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.408931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.540257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:45.548632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-447cd0ac-eeca-431e-9aae-814a5d0a5937 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.153517 Oct 12 02:32:45.552749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 281/556] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:45 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:45.586793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:45.587446 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:45.594427 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.594902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.595329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.596018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.596448 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.596897 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.753151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:45.762146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-35bf0f40-68e4-4972-b723-a6bbd547effb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.179056 Oct 12 02:32:45.762799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 276/557] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:45 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:45.858832 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:45.859706 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:45.866523 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.867060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.867633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.868434 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.868994 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.869654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.903983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:45.904746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:45.914588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.915148 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.915946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.916645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:45.917116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:45.917536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:45.954543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:45.963795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b42b97cd-857c-4cd1-9055-711b2e8c4a37 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.109803 Oct 12 02:32:45.964635 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 282/558] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:45 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:46.072223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:46.082492 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:46.084079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:46.085257 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-39121b80-3317-436d-81cf-94bfa05b7712 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.185967 Oct 12 02:32:46.085816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 277/559] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:45 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:46.574044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:46.575516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:46.584503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:46.585188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:46.585883 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:46.586706 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:46.587648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:46.588179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:46.742170 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:46.752003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7ad68cf1-9e68-4abf-a225-5539362c115f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.184135 Oct 12 02:32:46.752662 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 283/560] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:46 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:46.787740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:46.788352 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:46.795144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:46.795611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:46.796222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:46.796958 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:46.797414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:46.797833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:46.946624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:46.958269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f0a07c5f-a116-476d-99dc-cc7d79b5b691 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.174231 Oct 12 02:32:46.959902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 278/561] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:46 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:46.987391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:46.988219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:46.996690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:46.997541 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:46.997981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:46.998862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:46.999316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:46.999738 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:47.074934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:47.082608 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2bb5ae72-1689-47fe-aeec-1119531e7ec0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.099322 Oct 12 02:32:47.083189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 284/562] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:46 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:47.113339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:47.114392 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:47.121380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:47.122149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:47.122926 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:47.123479 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:47.123927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:47.124339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:47.263725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:47.282025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:47.283618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:47.284830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5b6b8724-409f-49df-bbbf-6a34f926cad4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.176073 Oct 12 02:32:47.285570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 279/563] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:47 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:47.790961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:47.792834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:47.800883 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:47.801763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:47.802761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:47.803882 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:47.804591 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:47.805690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:47.981484 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:47.982097 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:47.989112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:47.989600 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:47.990084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:47.990879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:47.991396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:47.991815 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:48.041105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:48.048864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3f75e1fb-1b74-49bc-a6a1-4b392202c10c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.277096 Oct 12 02:32:48.049615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 285/564] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:47 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:48.109808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:48.110663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:48.117406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:48.118132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:48.118712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:48.119743 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:48.121119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:48.121709 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:48.132079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:48.140814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7bdb693f-c8bc-4646-a6f5-94e34e34428f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.163280 Oct 12 02:32:48.141934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 280/565] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:47 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:48.199444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:48.207750 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5b2df579-ac72-43cc-bf9d-41c40772e1cc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.105782 Oct 12 02:32:48.208326 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 286/566] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:48 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:48.312163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:48.312771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:48.322251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:48.322927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:48.323517 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:48.324456 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:48.325092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:48.325670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:48.494896 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:48.505441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:48.507337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:48.509022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ba760349-839b-45d6-8f0b-2d2fa6a8fa7f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.201152 Oct 12 02:32:48.510672 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 281/567] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:48 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:49.072397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:49.074881 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:49.083215 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.084086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.084792 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.086213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.086904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.087548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.163807 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:49.164488 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:49.171790 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.172561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.173768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.174823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.175685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.176531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.181224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:49.316636 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:49.332198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ecb3e63e-cfb3-470b-8fb5-51da5ba16091 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.264516 Oct 12 02:32:49.333503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 287/568] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:49 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:49.342388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:49.348151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:49.349015 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:49.355969 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.357038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.357669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.358557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.359170 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.359749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.359863 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ac8390ad-da72-43c9-afd3-d033d4865508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.199888 Oct 12 02:32:49.360372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 282/569] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:49 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:49.442824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:49.451428 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-623a4593-bd18-4011-9625-14d0ce1e99b7 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.116425 Oct 12 02:32:49.452692 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 288/570] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:49 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:49.535977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:49.537022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:49.544024 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.544378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.545014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.545723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:49.546181 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:49.546623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:49.694470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:49.707059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:49.709172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:49.710361 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-28d52b7f-5a09-4d2e-9552-da9d71023f68 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.178788 Oct 12 02:32:49.711161 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 283/571] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:49 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:50.356633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:50.357275 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:50.370258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.373211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.375026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.375799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.376270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.380331 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.384403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:50.385059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:50.395530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.396031 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.396442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.397222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.397618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.398039 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.584300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:50.594635 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ae712ab6-f8bf-41d1-b384-5848c8c166fb tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1366 microversion: 2.1 time: 0.214643 Oct 12 02:32:50.595229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 284/572] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:50 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1366 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:50.599963 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:50.600571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:50.608051 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.608559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.609060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.609754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.610204 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.610624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.693101 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:50.706925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-78aecbe8-c196-478c-88aa-faff685642f5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.355213 Oct 12 02:32:50.730761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 289/573] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:50 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:50.731083 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:50.732744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-540984d7-05d6-4a26-930d-c4260545587b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.136336 Oct 12 02:32:50.734295 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 285/574] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:50 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:50.748926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:50.749537 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:50.755427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.755913 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.756341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.759963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:50.760955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:50.761330 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:50.974358 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:50.987373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:50.988939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:50.990118 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6f15e65a-4040-4e3b-bdc9-a3e29e037659 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.247148 Oct 12 02:32:50.990678 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 290/575] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:50 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:51.622649 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:51.623517 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:51.630435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.630937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.631355 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:51.632053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.632499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.632965 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:51.740336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:51.741069 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:51.749752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.753459 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.754108 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:51.755095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.756559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.757258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:51.787732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:51.797301 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-cac44548-f605-4ffa-bc05-e6bc8987f508 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1556 microversion: 2.1 time: 0.179090 Oct 12 02:32:51.798144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 286/576] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:51 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1556 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:51.808385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:51.813174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:51.820645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.821269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.822207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:51.823304 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.823860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.824440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:51.910037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:51.921400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6fed09d0-a744-4bcb-91a3-9530747702b6 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1270 microversion: 2.1 time: 0.121918 Oct 12 02:32:51.923185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 287/577] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:51 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1270 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:51.928488 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:51.928945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:51.935249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.936040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.937201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:51.937904 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:51.938358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:51.938774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:52.014726 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:52.023262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c7dcf379-efdd-4602-b54c-66899fd2b9aa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.288013 Oct 12 02:32:52.024793 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 291/578] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:51 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:52.032658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:52.033467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:52.048233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-420fb7d2-18ed-4dd5-8cb5-09e7c4c7c244 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.123440 Oct 12 02:32:52.048749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:52.049411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:52.049998 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:52.050874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:52.051479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:52.052068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:52.053927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 288/579] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:32:51 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e/os-interface => generated 284 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:52.290457 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:52.305035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:52.306394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:52.312919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a73df2ba-f997-4c4d-8242-7cf73296fe53 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.284722 Oct 12 02:32:52.328852 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 292/580] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:52 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:52.949223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:52.950527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:52.962125 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:52.963772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:52.967085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:52.969274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:52.970022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:52.971614 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:53.054660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:53.055291 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:53.072355 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:53.072889 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:53.074072 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:53.074981 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:53.076012 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:53.076610 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:53.101058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:53.111290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8cae63e1-32f3-42c0-bb92-8dbf1ac729db tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1278 microversion: 2.1 time: 0.168854 Oct 12 02:32:53.114927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 289/581] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:52 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1278 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:53.343531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:53.344393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:53.358528 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:53.360416 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:53.361336 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:53.362521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:53.363172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:53.363788 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:53.458775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:53.471651 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b862af0c-1ab3-422f-89f7-a38548b80709 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.426960 Oct 12 02:32:53.498733 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 293/582] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:53 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:53.698918 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:53.708434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:53.709913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:53.722884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d9a1865f-54dd-4d04-a6ed-c4d9f06128e7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.383415 Oct 12 02:32:53.723717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 290/583] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:53 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:32:54.022572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-659d7cb1-61c9-4a44-9b86-79080355f768 req-498096b0-2726-42a2-a812-61eff807fd48 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7dc118ab-0194-4cbb-b335-caf4c25cca7e", "tag": "79216b85-6ffe-4aa2-b822-11695d819ed3"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:54.050573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-659d7cb1-61c9-4a44-9b86-79080355f768 req-498096b0-2726-42a2-a812-61eff807fd48 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:54.052991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-659d7cb1-61c9-4a44-9b86-79080355f768 req-498096b0-2726-42a2-a812-61eff807fd48 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:54.054828 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-659d7cb1-61c9-4a44-9b86-79080355f768 req-498096b0-2726-42a2-a812-61eff807fd48 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:54.109244 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-659d7cb1-61c9-4a44-9b86-79080355f768 req-498096b0-2726-42a2-a812-61eff807fd48 service nova] Creating event network-changed:79216b85-6ffe-4aa2-b822-11695d819ed3 for instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e on hv-cinder-84914 Oct 12 02:32:54.124408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-659d7cb1-61c9-4a44-9b86-79080355f768 req-498096b0-2726-42a2-a812-61eff807fd48 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.114231 Oct 12 02:32:54.125236 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 294/584] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:54.137367 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:54.138842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:54.148227 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:54.148746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:54.154563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:54.155544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:54.157442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:54.159137 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:54.298378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:54.316146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-502f645c-21d6-4105-a2ee-0ea209fcd677 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1276 microversion: 2.1 time: 0.184543 Oct 12 02:32:54.320888 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 291/585] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:54 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1276 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:54.512380 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:54.513374 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:54.529832 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:54.533085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:54.536404 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:54.541181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:54.543696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:54.545796 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:54.755182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:54.756185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:54.768452 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:54.769227 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:54.770010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:54.772098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:54.772912 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:54.773491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:55.003659 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:55.018019 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c4c13fa7-b05a-459d-bfcf-8d509e96edd3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1371 microversion: 2.1 time: 0.511712 Oct 12 02:32:55.021722 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 295/586] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:54 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1371 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:55.133457 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:55.145424 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:55.152854 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:55.154195 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bf39a968-0396-4355-b943-60e77c229894 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.405674 Oct 12 02:32:55.169914 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 292/587] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:54 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:55.352013 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:55.352611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:55.371658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:55.372400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:55.373078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:55.373963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:55.374554 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:55.375139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:55.639948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:55.646118 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:32:55.648419 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1faf4b09-8c0a-47ee-bb77-9e689d83f409 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1318 microversion: 2.1 time: 0.300127 Oct 12 02:32:55.649074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 296/588] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:55 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1318 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:56.047780 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:56.049041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:56.060395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.061187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.063615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.064531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.065200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.067190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.181940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:56.183442 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:56.195814 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.205428 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.206385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.207483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.208106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.208683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.396327 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:56.405302 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-89d1f220-6ad5-4213-9886-f9f8cf300969 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1566 microversion: 2.1 time: 0.361652 Oct 12 02:32:56.411396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 293/589] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:56 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1566 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:56.426357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-ae38f2a6-f51f-4afd-a456-bc284026dbe1 req-779c4af5-daa6-49dc-a50e-8544361f82d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48eb4bd4-2680-4fc6-85c0-dae85fb740fa", "name": "network-vif-plugged", "status": "completed", "tag": "0a31a866-f72c-4f7d-822f-7d54a4e3afec"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:56.448247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-ae38f2a6-f51f-4afd-a456-bc284026dbe1 req-779c4af5-daa6-49dc-a50e-8544361f82d0 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.449449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-ae38f2a6-f51f-4afd-a456-bc284026dbe1 req-779c4af5-daa6-49dc-a50e-8544361f82d0 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.449895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-ae38f2a6-f51f-4afd-a456-bc284026dbe1 req-779c4af5-daa6-49dc-a50e-8544361f82d0 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.474930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-ae38f2a6-f51f-4afd-a456-bc284026dbe1 req-779c4af5-daa6-49dc-a50e-8544361f82d0 service nova] Creating event network-vif-plugged:0a31a866-f72c-4f7d-822f-7d54a4e3afec for instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa on hv-cinder-84914 Oct 12 02:32:56.481814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-ae38f2a6-f51f-4afd-a456-bc284026dbe1 req-779c4af5-daa6-49dc-a50e-8544361f82d0 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069020 Oct 12 02:32:56.482272 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 294/590] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:56.493331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:56.499327 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:56.513711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.514956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.517809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.518843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.521865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.522507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.567662 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:56.585805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:56.591344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:56.592924 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-774585d8-0aff-4e7c-a57f-d0c1cc86de80 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.415204 Oct 12 02:32:56.600275 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 297/591] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:56 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:32:56.680079 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:56.681017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:56.688229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.688993 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.689572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.690521 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:56.691126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:56.691696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:56.785679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7e4c4bfa-0684-4c79-8d1a-94d8df07d1ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.298930 Oct 12 02:32:56.786456 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 295/592] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:32:56 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-interface => generated 285 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:57.026113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-16b91235-787e-4755-8756-d5afbc81ea66 req-ba7968e2-bb85-4a51-9dd2-89556f9d0fd8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e624aca8-38d2-45b4-a145-ad8f496a137e", "tag": "2aadefe1-189a-4779-9014-9966dfb12cda"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:57.041244 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-16b91235-787e-4755-8756-d5afbc81ea66 req-ba7968e2-bb85-4a51-9dd2-89556f9d0fd8 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:57.041700 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-16b91235-787e-4755-8756-d5afbc81ea66 req-ba7968e2-bb85-4a51-9dd2-89556f9d0fd8 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:57.042677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-16b91235-787e-4755-8756-d5afbc81ea66 req-ba7968e2-bb85-4a51-9dd2-89556f9d0fd8 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:57.067779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-16b91235-787e-4755-8756-d5afbc81ea66 req-ba7968e2-bb85-4a51-9dd2-89556f9d0fd8 service nova] Creating event network-changed:2aadefe1-189a-4779-9014-9966dfb12cda for instance e624aca8-38d2-45b4-a145-ad8f496a137e on hv-cinder-84914 Oct 12 02:32:57.079723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-16b91235-787e-4755-8756-d5afbc81ea66 req-ba7968e2-bb85-4a51-9dd2-89556f9d0fd8 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059045 Oct 12 02:32:57.083294 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 296/593] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:32:57.200541 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:57.212435 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-808f5d4d-c406-4b6d-a5bb-47b4e3d2c497 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1318 microversion: 2.1 time: 0.543346 Oct 12 02:32:57.214014 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 298/594] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:56 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1318 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:32:57.630101 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:57.630794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:57.643874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:57.644258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:57.644685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:57.645398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:57.648803 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:57.649438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:57.848165 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:57.858777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:57.860198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:57.862366 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3016b985-0369-4ec7-8c98-a63071c73b9e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.236594 Oct 12 02:32:57.863487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 297/595] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:57 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:58.238192 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:58.238815 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:58.246043 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:58.247129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:58.247777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:58.248494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:58.248977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:58.249412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:58.631104 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:58.643461 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0a743eb9-28fd-43fa-8b3e-23022331e687 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1318 microversion: 2.1 time: 0.410132 Oct 12 02:32:58.644278 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 299/596] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:58 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1318 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:32:58.888836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:58.890178 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:58.899913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:58.900726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:58.901360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:58.902392 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:58.903001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:58.903580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:59.138190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:59.149243 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-1df79d31-23dd-4d43-a22c-c66e49afcb3b req-99fada5c-2d8a-4765-b470-6cff54cea12a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac5527bc-4e21-4157-9415-41e66105dc9c", "tag": "3c705692-0c55-4ed6-88ed-b7ba46d3442f"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:32:59.150263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:59.152283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:32:59.153523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-00d3589f-f73c-4588-82c0-2f738b932818 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.269074 Oct 12 02:32:59.154581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 298/597] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:32:58 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:32:59.169051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-1df79d31-23dd-4d43-a22c-c66e49afcb3b req-99fada5c-2d8a-4765-b470-6cff54cea12a service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:59.169762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-1df79d31-23dd-4d43-a22c-c66e49afcb3b req-99fada5c-2d8a-4765-b470-6cff54cea12a service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:59.174359 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-1df79d31-23dd-4d43-a22c-c66e49afcb3b req-99fada5c-2d8a-4765-b470-6cff54cea12a service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:59.187698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-1df79d31-23dd-4d43-a22c-c66e49afcb3b req-99fada5c-2d8a-4765-b470-6cff54cea12a service nova] Creating event network-changed:3c705692-0c55-4ed6-88ed-b7ba46d3442f for instance ac5527bc-4e21-4157-9415-41e66105dc9c on hv-cinder-84914 Oct 12 02:32:59.193218 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-1df79d31-23dd-4d43-a22c-c66e49afcb3b req-99fada5c-2d8a-4765-b470-6cff54cea12a service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049916 Oct 12 02:32:59.194142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 300/598] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:32:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:32:59.673199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:32:59.673199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:32:59.685243 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:59.691791 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:59.692283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:32:59.693139 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:32:59.693691 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:32:59.695634 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:00.103409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:00.115532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ffc2917b-bc6e-4d74-a10b-68b18452a0c8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1318 microversion: 2.1 time: 0.451522 Oct 12 02:33:00.117701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 299/599] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:32:59 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1318 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:00.184628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:00.186320 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:00.194014 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:00.194760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:00.195362 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:00.196496 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:00.197192 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:00.197768 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:00.478420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:00.491184 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:00.492631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:00.523374 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-26837372-f208-4a26-a1b6-e811d941f504 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.334276 Oct 12 02:33:00.524025 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 301/600] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:33:00 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:33:01.139991 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:01.140582 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:01.146794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:01.147365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:01.147964 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:01.148844 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:01.149462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:01.150023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:01.355736 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:01.362497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:01.384741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d16787af-1022-4cfb-8d3e-efe5179649d5 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1318 microversion: 2.1 time: 0.248995 Oct 12 02:33:01.386211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 300/601] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:01 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1318 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:01.555174 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:01.555872 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:01.584205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:01.588441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:01.592573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:01.593595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:01.594310 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:01.597240 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:01.820292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:01.829717 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:01.832012 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:01.833375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6a35bade-ee3a-42b3-a186-6c07176dd4d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1808 microversion: 2.20 time: 0.284023 Oct 12 02:33:01.834659 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 302/602] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:33:01 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1808 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:33:02.073879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-95f68804-6b1c-422d-a0ab-b25802e80000 req-335eb717-62b3-4abf-b521-06d57b6e0c54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e624aca8-38d2-45b4-a145-ad8f496a137e", "name": "network-vif-plugged", "status": "completed", "tag": "2aadefe1-189a-4779-9014-9966dfb12cda"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:02.085401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-95f68804-6b1c-422d-a0ab-b25802e80000 req-335eb717-62b3-4abf-b521-06d57b6e0c54 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:02.085883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-95f68804-6b1c-422d-a0ab-b25802e80000 req-335eb717-62b3-4abf-b521-06d57b6e0c54 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:02.086357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-95f68804-6b1c-422d-a0ab-b25802e80000 req-335eb717-62b3-4abf-b521-06d57b6e0c54 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:02.099732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-95f68804-6b1c-422d-a0ab-b25802e80000 req-335eb717-62b3-4abf-b521-06d57b6e0c54 service nova] Creating event network-vif-plugged:2aadefe1-189a-4779-9014-9966dfb12cda for instance e624aca8-38d2-45b4-a145-ad8f496a137e on hv-cinder-84914 Oct 12 02:33:02.106263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-95f68804-6b1c-422d-a0ab-b25802e80000 req-335eb717-62b3-4abf-b521-06d57b6e0c54 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036255 Oct 12 02:33:02.106852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 301/603] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:02.412585 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:02.416177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:02.429726 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:02.430218 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:02.430639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:02.431336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:02.431774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:02.433136 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:02.580567 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:02.591164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ea657f80-ba4f-423e-86b7-de4bfc06c27b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1318 microversion: 2.1 time: 0.184753 Oct 12 02:33:02.591729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 303/604] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:02 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1318 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:02.859894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:02.860581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:02.871336 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:02.871741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:02.872181 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:02.872960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:02.873425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:02.873844 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:03.065658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:03.076669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:03.078500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:03.080032 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d017599c-3394-4980-b5db-456858d5af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1795 microversion: 2.20 time: 0.224364 Oct 12 02:33:03.081091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 302/605] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:33:02 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1795 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:33:03.614379 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:03.615515 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:03.618206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=88 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:03.625788 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:03.626279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:03.626708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:03.632745 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:03.633227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:03.633654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:03.810217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:03.819000 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bfddb4f8-bae1-45ce-8e12-99e6525248fc tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1318 microversion: 2.1 time: 0.208590 Oct 12 02:33:03.819613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 304/606] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:03 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1318 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:04.843607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:04.844413 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:04.853504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:04.854058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:04.854588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:04.855579 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:04.856258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:04.856999 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:05.008690 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:05.018436 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-79407d06-1670-4aa8-bd4c-6a2a0eb93d13 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1505 microversion: 2.1 time: 0.178295 Oct 12 02:33:05.029650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 303/607] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:04 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1505 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:05.044739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e328a2f5-a1fe-4733-a3a6-c0abb303d6f5", "device": "/dev/sdb"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:05.047881 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:05.054808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:05.055498 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:05.057214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:05.058141 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:05.058634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:05.059098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:05.388573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-306d222f-b2c0-4877-b3da-d91de1bdd122 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "POST /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.347873 Oct 12 02:33:05.389336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 305/608] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:33:05 2022] POST /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e/os-volume_attachments => generated 194 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:33:09.706085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-572a3d5c-51fc-4542-bd3c-4ac4e14b9251 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Action: 'create', calling method: .version_select at 0x7f59d375a8b0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1657289250", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBf/K6GmSM29tU/nRW+gAkzpHrZ+Fc1KpFs/dSg8ct6dbfQIYziiGX7j+dwIRNw2IHewdtdxdfYOiDM66PBh07Z6xNiuw/DQxe/Y4wptkzIgLS/dJSpl0SK7bAo/G2uLuA=="}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:09.711829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=83 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:09.723300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-572a3d5c-51fc-4542-bd3c-4ac4e14b9251 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Getting quotas for project c57365923b844fe9b29d9e223ad49ce4. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:33:09.727037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-572a3d5c-51fc-4542-bd3c-4ac4e14b9251 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Getting quotas for user 945de18174894cd683ebf34372e845f0 and project c57365923b844fe9b29d9e223ad49ce4. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:33:09.748045 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-572a3d5c-51fc-4542-bd3c-4ac4e14b9251 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.045877 Oct 12 02:33:09.748619 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 304/609] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:33:09 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:09.799557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-903183345", "imageRef": "34f77620-7110-4faf-9ebc-d6e1bdf8b57f", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1657289250", "networks": [{"uuid": "29d9435c-4eff-4eb3-8183-9024706bdf4d"}]}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:09.851880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] validate_networks() for [('29d9435c-4eff-4eb3-8183-9024706bdf4d', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:33:10.214011 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:33:10.214431 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:33:10.214897 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:33:10.215298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:33:10.215699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:33:10.224238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:10.224746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:10.225182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:10.225905 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:10.226358 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:10.226780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:10.243097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Getting quotas for project c57365923b844fe9b29d9e223ad49ce4. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:33:10.247022 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Getting quotas for user 945de18174894cd683ebf34372e845f0 and project c57365923b844fe9b29d9e223ad49ce4. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:33:10.257728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:33:10.260471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='34f77620-7110-4faf-9ebc-d6e1bdf8b57f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:33:10.305772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-61c466ef-7067-496d-815e-00436566be50 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.511004 Oct 12 02:33:10.306412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 306/610] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:33:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 512 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:33:10.317302 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9255ec54-d8f3-411f-a72d-5b43b6bd1652 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:10.317912 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9255ec54-d8f3-411f-a72d-5b43b6bd1652 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:10.371396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9255ec54-d8f3-411f-a72d-5b43b6bd1652 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:10.378407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9255ec54-d8f3-411f-a72d-5b43b6bd1652 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1258 microversion: 2.1 time: 0.065040 Oct 12 02:33:10.378982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 305/611] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:10 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1258 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:11.402183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:11.402702 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:11.409661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:11.410138 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:11.410562 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:11.411322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:11.411796 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:11.412217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:11.471908 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:11.479949 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a1529b46-9afc-4aa1-a0a6-e5e0528e5bc3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1310 microversion: 2.1 time: 0.082249 Oct 12 02:33:11.480508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 307/612] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:11 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1310 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:12.503278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:12.503937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:12.510988 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:12.511469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:12.511900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:12.512573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:12.513045 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:12.513469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:12.572323 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:12.579869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a7f16619-a55f-493a-b5d8-28295acdedb3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1318 microversion: 2.1 time: 0.080451 Oct 12 02:33:12.580428 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 306/613] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:12 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1318 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:13.605269 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:13.606977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:13.616687 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:13.618043 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:13.618855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:13.620563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:13.621306 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:13.621951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:13.782126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:13.790148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b350177f-e882-4250-b0a3-7a093ef35112 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.190511 Oct 12 02:33:13.791639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 308/614] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:13 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:14.817895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:14.818711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:14.824942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:14.825532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:14.825959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:14.826656 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:14.827105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:14.827525 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:14.963864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:14.971727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a198356e-9963-45d2-b45e-def7117e47a9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.157728 Oct 12 02:33:14.972284 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 307/615] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:14 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:15.994061 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:15.994658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:16.001705 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:16.002187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:16.002623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:16.003311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:16.003774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:16.004175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:16.144500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:16.154934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d89657b5-0390-40b9-88a5-8277ea1955d8 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.164598 Oct 12 02:33:16.156191 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 309/616] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:15 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:17.179602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:17.180196 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:17.187336 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:17.188012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:17.188608 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:17.189357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:17.189771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:17.190191 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:17.363639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:17.373098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-18714a14-c9c2-40db-8829-7cfe764c33a7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.197115 Oct 12 02:33:17.373890 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 308/617] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:17 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:33:18.398676 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:18.399433 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:18.407030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:18.407534 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:18.407966 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:18.408669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:18.409152 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:18.411348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:18.549272 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:18.556836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8fc71345-9573-4f5a-8597-4da18f99733f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.162045 Oct 12 02:33:18.557408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 310/618] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:18 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:19.579459 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:19.580068 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:19.586384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:19.587171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:19.587891 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:19.588597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:19.589080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:19.590005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:19.723752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:19.731576 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c2542270-201f-44ad-ab71-583e90261ecb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.156005 Oct 12 02:33:19.732139 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 309/619] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:19 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:20.754676 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:20.755282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:20.757710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:20.758375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=6 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:20.764413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:20.764920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:20.765353 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:20.766040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:20.766488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:20.766906 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:20.896655 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:20.904838 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-43e8d27a-37e3-4cef-9a33-b02fe2fe9570 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.153968 Oct 12 02:33:20.905478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 311/620] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:20 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:21.927689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:21.928424 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:21.930414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=6 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:21.937267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:21.937788 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:21.938219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:21.938941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:21.939387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:21.939801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:22.073632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:22.082450 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fdd372c0-ebb9-4bd8-9727-906682f57187 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.158644 Oct 12 02:33:22.083028 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 310/621] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:21 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:33:23.104865 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:23.106173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:23.114443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:23.115111 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:23.115611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:23.116388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:23.116938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:23.117439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:23.254001 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:23.261765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-306e2518-e4be-49e0-8497-6f09857d1561 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.160884 Oct 12 02:33:23.262975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 312/622] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:23 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:24.285855 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:24.286457 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:24.292310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:24.292816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:24.293289 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:24.293959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:24.294401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:24.294814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:24.428446 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:24.436234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-23136747-3bd0-48e6-9c22-f2d3fcf2bd04 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.154222 Oct 12 02:33:24.436844 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 311/623] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:24 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:24.854155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:24.854655 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:24.861419 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:24.862014 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:24.862482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:24.863181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:24.863633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:24.864050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:24.882030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:24.951118 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lazy-loading 'services' on Instance uuid e624aca8-38d2-45b4-a145-ad8f496a137e {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:33:25.026977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-956a42f0-6c57-475d-b64c-6e9a1bdafa5c tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "DELETE /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e/os-volume_attachments/e328a2f5-a1fe-4733-a3a6-c0abb303d6f5" status: 202 len: 0 microversion: 2.1 time: 0.176522 Oct 12 02:33:25.027619 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 313/624] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:33:24 2022] DELETE /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e/os-volume_attachments/e328a2f5-a1fe-4733-a3a6-c0abb303d6f5 => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:33:25.228156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-524b919f-625c-4885-ae61-e39f74c09571 req-535b0cea-71af-43c6-aba6-555b9201e824 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bacd4ab7-51ad-4943-9d16-92bd1fc1adfa", "tag": "299443a4-87d4-4257-8710-d074d71d9851"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:25.238218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-524b919f-625c-4885-ae61-e39f74c09571 req-535b0cea-71af-43c6-aba6-555b9201e824 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:25.238684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-524b919f-625c-4885-ae61-e39f74c09571 req-535b0cea-71af-43c6-aba6-555b9201e824 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:25.239106 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-524b919f-625c-4885-ae61-e39f74c09571 req-535b0cea-71af-43c6-aba6-555b9201e824 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:25.250438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-524b919f-625c-4885-ae61-e39f74c09571 req-535b0cea-71af-43c6-aba6-555b9201e824 service nova] Creating event network-changed:299443a4-87d4-4257-8710-d074d71d9851 for instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa on hv-cinder-84914 Oct 12 02:33:25.255277 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-524b919f-625c-4885-ae61-e39f74c09571 req-535b0cea-71af-43c6-aba6-555b9201e824 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030974 Oct 12 02:33:25.255856 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 312/625] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:25.459824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:25.460545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:25.467139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:25.467614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:25.468044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:25.468762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:25.469212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:25.469661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:25.594989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:25.603203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-096c00dd-6f84-4e1f-8ed0-ca1ae8583031 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.147555 Oct 12 02:33:25.603761 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 314/626] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:25 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:26.627427 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:26.628151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:26.634204 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:26.634682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:26.635111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:26.635787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:26.636237 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:26.636655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:26.766605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:26.774216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-01515f08-600c-4dee-8552-a290c142f65c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.150939 Oct 12 02:33:26.774774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 313/627] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:26 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:27.215550 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:27.216058 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:27.222412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:27.222892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:27.223318 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:27.223995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:27.224441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:27.224899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:27.250565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:33:27.322725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e8ab928f-c4e7-4416-bc71-18bcc90b4106 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "DELETE /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 204 len: 0 microversion: 2.1 time: 0.110756 Oct 12 02:33:27.323307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 315/628] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:33:27 2022] DELETE /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:33:27.331448 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:27.332089 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:27.338030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:27.338512 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:27.338943 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:27.339622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:27.340075 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:27.340496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:27.469197 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:27.476889 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e7122aa3-870b-43ef-b8da-fa3981b81c10 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1511 microversion: 2.1 time: 0.149248 Oct 12 02:33:27.477523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 314/629] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:27 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1511 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:27.798473 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:27.799180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:27.805107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:27.805592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:27.806025 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:27.806712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:27.807167 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:27.807591 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:27.933028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:27.940780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c82cc5e6-14ac-4e43-8a05-9f3d2612a73f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.146630 Oct 12 02:33:27.941353 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 316/630] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:27 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:28.501242 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:28.501849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:28.507759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:28.508227 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:28.508650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:28.509341 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:28.509781 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:28.510192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:28.644650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:28.652329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-162b0fdd-14f2-43bf-8374-8280c5089fa0 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1511 microversion: 2.1 time: 0.154903 Oct 12 02:33:28.652925 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 315/631] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:28 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1511 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:28.963462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:28.964086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:28.970472 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:28.970946 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:28.971370 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:28.972040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:28.972485 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:28.972971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:29.109764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:29.117622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4d53ed40-2855-40db-829f-a1fd387296f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.157974 Oct 12 02:33:29.118509 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 317/632] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:28 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:29.675373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:29.675893 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:29.681761 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:29.682234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:29.682655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:29.683323 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:29.683764 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:29.684174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:29.811003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:29.818480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fa0ed208-6954-4495-94e9-7cf82a5890a3 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1511 microversion: 2.1 time: 0.147045 Oct 12 02:33:29.819030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 316/633] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:29 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1511 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:30.143494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:30.144120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:30.150387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:30.150856 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:30.151281 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:30.151952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:30.152394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:30.152837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:30.368864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:30.376972 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8b03c456-b72d-44c4-b142-8a6072e19daf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.237330 Oct 12 02:33:30.377550 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 318/634] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:30 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:30.605715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-3d84d0b8-892d-4a0f-ad22-f0eff5593000 req-d91db630-427a-44f9-82c3-f001ddf891a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e624aca8-38d2-45b4-a145-ad8f496a137e", "name": "network-vif-deleted", "tag": "2aadefe1-189a-4779-9014-9966dfb12cda"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:30.617062 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-3d84d0b8-892d-4a0f-ad22-f0eff5593000 req-d91db630-427a-44f9-82c3-f001ddf891a6 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:30.617640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-3d84d0b8-892d-4a0f-ad22-f0eff5593000 req-d91db630-427a-44f9-82c3-f001ddf891a6 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:30.618164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-3d84d0b8-892d-4a0f-ad22-f0eff5593000 req-d91db630-427a-44f9-82c3-f001ddf891a6 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:30.630494 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-3d84d0b8-892d-4a0f-ad22-f0eff5593000 req-d91db630-427a-44f9-82c3-f001ddf891a6 service nova] Creating event network-vif-deleted:2aadefe1-189a-4779-9014-9966dfb12cda for instance e624aca8-38d2-45b4-a145-ad8f496a137e on hv-cinder-84914 Oct 12 02:33:30.636056 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-3d84d0b8-892d-4a0f-ad22-f0eff5593000 req-d91db630-427a-44f9-82c3-f001ddf891a6 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033948 Oct 12 02:33:30.637752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 317/635] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:30.726249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-ac454f74-9486-420a-a474-2b7f0a2cb42d req-11f2e943-8c62-4fac-a9c7-5a163b20e60f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bacd4ab7-51ad-4943-9d16-92bd1fc1adfa", "name": "network-vif-plugged", "status": "completed", "tag": "299443a4-87d4-4257-8710-d074d71d9851"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:30.738194 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-ac454f74-9486-420a-a474-2b7f0a2cb42d req-11f2e943-8c62-4fac-a9c7-5a163b20e60f service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:30.738847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-ac454f74-9486-420a-a474-2b7f0a2cb42d req-11f2e943-8c62-4fac-a9c7-5a163b20e60f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:30.739341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-ac454f74-9486-420a-a474-2b7f0a2cb42d req-11f2e943-8c62-4fac-a9c7-5a163b20e60f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:30.755272 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-ac454f74-9486-420a-a474-2b7f0a2cb42d req-11f2e943-8c62-4fac-a9c7-5a163b20e60f service nova] Creating event network-vif-plugged:299443a4-87d4-4257-8710-d074d71d9851 for instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa on hv-cinder-84914 Oct 12 02:33:30.760114 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-ac454f74-9486-420a-a474-2b7f0a2cb42d req-11f2e943-8c62-4fac-a9c7-5a163b20e60f service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038525 Oct 12 02:33:30.761336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 319/636] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:30.842708 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:30.843541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:30.849802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:30.850314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:30.850661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:30.851349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:30.851799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:30.852290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:30.884774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:30.885252 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:30.885683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:30.886122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lazy-loading 'fault' on Instance uuid e624aca8-38d2-45b4-a145-ad8f496a137e {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:33:30.928315 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:30.937156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1adecd54-4a6c-478a-b044-3c0acfb3f98b tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1308 microversion: 2.1 time: 0.098756 Oct 12 02:33:30.938001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 318/637] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:30 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1308 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:31.400834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:31.401436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:31.407696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:31.408174 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:31.408603 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:31.409425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:31.409904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:31.410322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:31.536337 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:31.543883 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c49b1038-1bb4-4bbf-ae0a-27214c313004 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.148260 Oct 12 02:33:31.544447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 320/638] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:31 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:31.962439 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:31.963045 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:31.969242 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:31.969778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:31.970215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:31.970902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:31.971385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:31.971809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:31.975835 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=87 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:32.003337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:32.003818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:32.004249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:32.004689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lazy-loading 'fault' on Instance uuid e624aca8-38d2-45b4-a145-ad8f496a137e {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:33:32.048506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bbcc7dcbf32b4048879bbc77e0998b52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81144c673e7b44dcb3b0f700ee628719', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:32.056217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bb7cdb22-5674-4a7a-81a9-3a9ec2b26bc1 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 200 len: 1308 microversion: 2.1 time: 0.097659 Oct 12 02:33:32.056797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 319/639] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:31 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 1308 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:32.568724 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:32.569388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:32.576551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:32.577182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:32.577628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:32.578338 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:32.578797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:32.579219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:32.710314 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:32.718618 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-180b967c-105b-418b-af3d-6f1d04a09f82 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1358 microversion: 2.1 time: 0.155603 Oct 12 02:33:32.719178 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 321/640] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:32 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:33:33.081714 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:33.082319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] [instance: e624aca8-38d2-45b4-a145-ad8f496a137e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:33.088249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:33.088753 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:33.089185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:33.089945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:33.090389 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:33.090803 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:33.104032 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] HTTP exception thrown: Instance e624aca8-38d2-45b4-a145-ad8f496a137e could not be found. Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Request method failure captured: Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e HTTP/1.1 Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRibAWv4KbWVc6HhsH9Vu4uASpvDQ2d8uwxitHHc61pAl5Z0K_Iq5LjdO1RWnXS9a92ffwo8-SE9WT3KRO52id2ZHDNXujgbasBq52COoc50xCZeJmm9noLC6-V3TEpp-EMGvYZoXyN8UzwCLzTdWD1NgoQhANTntnAuZC1b0Wy-Q98A Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 81144c673e7b44dcb3b0f700ee628719 Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1936448719 Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:33:33.105713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/81144c673e7b44dcb3b0f700ee628719"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/81144c673e7b44dcb3b0f700ee628719"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/81144c673e7b44dcb3b0f700ee628719"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_81144c673e7b44dcb3b0f700ee628719", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-VolumesSnapshotTestJSON-1936448719 Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 81144c673e7b44dcb3b0f700ee628719 Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1936448719 Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-VolumesSnapshotTestJSON-1936448719-project Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: bbcc7dcbf32b4048879bbc77e0998b52 Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-VolumesSnapshotTestJSON-1936448719-project Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance e624aca8-38d2-45b4-a145-ad8f496a137e could not be found. Oct 12 02:33:33.107211 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'e624aca8-38d2-45b4-a145-ad8f496a137e'} Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e624aca8-38d2-45b4-a145-ad8f496a137e could not be found. Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:33:33.108085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e624aca8-38d2-45b4-a145-ad8f496a137e could not be found. Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e624aca8-38d2-45b4-a145-ad8f496a137e could not be found. Oct 12 02:33:33.109547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:33.111073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] Returning 404 to user: Instance e624aca8-38d2-45b4-a145-ad8f496a137e could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:33:33.111073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1f7bc0f2-5a31-4668-8884-4e2b466fc7d8 tempest-VolumesSnapshotTestJSON-1936448719 tempest-VolumesSnapshotTestJSON-1936448719-project] 10.222.0.23 "GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e" status: 404 len: 111 microversion: 2.1 time: 0.029199 Oct 12 02:33:33.111073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 320/641] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:33 2022] GET /compute/v2.1/servers/e624aca8-38d2-45b4-a145-ad8f496a137e => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:33:33.741259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:33.741895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:33.744274 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=54,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:33.749365 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:33.749843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:33.750274 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:33.750954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:33.751405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:33.751830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:33.880136 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:33.888366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-97848da3-bbe2-4c0a-abda-ecfed1d4644c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1547 microversion: 2.1 time: 0.150971 Oct 12 02:33:33.888960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 322/642] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:33 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1547 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:33.910448 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:33.911060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:33.916995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:33.917982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:33.918430 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:33.919107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:33.919556 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:33.919974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:34.046563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:34.054114 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e7f564db-26bd-4988-811e-e7d63643eb4d tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1547 microversion: 2.1 time: 0.147465 Oct 12 02:33:34.054666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 321/643] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:33 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1547 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:34.075785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:34.080730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:34.081481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Searching by: {'deleted': False, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4'} {{(pid=118864) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 12 02:33:34.087897 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:34.088366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:34.088825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:34.089540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:34.089997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:34.090412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:34.179880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.multi_cell_list [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=118864) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 12 02:33:34.271847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-deb9248e-2035-4e03-b243-34e39b2fb744 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.199718 Oct 12 02:33:34.272356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 323/644] 10.222.0.23 () {58 vars in 1125 bytes} [Wed Oct 12 02:33:34 2022] GET /compute/v2.1/servers => generated 342 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:34.284361 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:34.285001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:34.290888 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:34.291368 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:34.291797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:34.292490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:34.292985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:34.293406 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:34.430423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:34.438983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f54468d3-0284-4304-8483-b3dc796c36ed tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1547 microversion: 2.1 time: 0.158377 Oct 12 02:33:34.439543 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 322/645] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:34 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1547 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:38.142464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "41cb5200-afd2-41c4-af92-d2fba5384e1a"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:38.144724 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:38.151196 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:38.151653 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:38.152203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:38.152952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:38.153420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:38.153848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:38.414540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-878302a9-66be-4bab-b8ba-4d7b02b8bceb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.275873 Oct 12 02:33:38.415259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 324/646] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:33:38 2022] POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments => generated 194 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:39.832320 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:39.832877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:39.834735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:39.841154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:39.841650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:39.842071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:39.842754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:39.843207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:39.843628 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:39.868773 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7ba4b72e-dee0-4acb-aa82-e37b5e9bd0dc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e/ips" status: 200 len: 149 microversion: 2.1 time: 0.040484 Oct 12 02:33:39.869328 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 323/647] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:33:39 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e/ips => generated 149 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:39.881826 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "960f293c-f2d1-48a5-8f68-2c9c685f2c9a"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:39.884332 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:39.890989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:39.891470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:39.891896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:39.892567 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:39.893032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:39.893446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:39.970392 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Invalid volume: volume 960f293c-f2d1-48a5-8f68-2c9c685f2c9a is already attached to instances: 76667a6b-65ab-4a9d-8af3-0440b0dad579 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: POST /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e/os-volume_attachments HTTP/1.1 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 74 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:33:39.972096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {"volumeAttachment": {"volumeId": "960f293c-f2d1-48a5-8f68-2c9c685f2c9a"}} Oct 12 02:33:39.973721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Invalid volume: volume 960f293c-f2d1-48a5-8f68-2c9c685f2c9a is already attached to instances: 76667a6b-65ab-4a9d-8af3-0440b0dad579 Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'server_id': '7dc118ab-0194-4cbb-b335-caf4c25cca7e', 'body': {'volumeAttachment': {'volumeId': '960f293c-f2d1-48a5-8f68-2c9c685f2c9a'}}} Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 960f293c-f2d1-48a5-8f68-2c9c685f2c9a is already attached to instances: 76667a6b-65ab-4a9d-8af3-0440b0dad579 Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:33:39.974612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 960f293c-f2d1-48a5-8f68-2c9c685f2c9a is already attached to instances: 76667a6b-65ab-4a9d-8af3-0440b0dad579 Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:39.975926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 960f293c-f2d1-48a5-8f68-2c9c685f2c9a is already attached to instances: 76667a6b-65ab-4a9d-8af3-0440b0dad579 Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 400 to user: Invalid volume: volume 960f293c-f2d1-48a5-8f68-2c9c685f2c9a is already attached to instances: 76667a6b-65ab-4a9d-8af3-0440b0dad579 {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9fe243d7-6279-4535-841e-d032373ad8ec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.096611 Oct 12 02:33:39.977298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 325/648] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:33:39 2022] POST /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e/os-volume_attachments => generated 174 bytes in 98 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:33:39.982417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:39.982908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:39.988991 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:39.989468 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:39.989902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:39.990585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:39.991039 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:39.991458 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:40.015923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:33:40.087995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2e6ca9b1-ff54-4507-9fdc-50aea29e0fc9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 204 len: 0 microversion: 2.1 time: 0.109072 Oct 12 02:33:40.088500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 324/649] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:33:39 2022] DELETE /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:33:40.097467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:40.098091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:40.104970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:40.105451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:40.105882 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:40.106604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:40.107050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:40.107466 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:40.241043 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:40.244538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:40.250429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b0ef324b-e2cc-4f38-8888-60321d9e40d6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1680 microversion: 2.1 time: 0.156943 Oct 12 02:33:40.250992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 326/650] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:40 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1680 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:41.275127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:41.275827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:41.282189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:41.282667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:41.283098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:41.283774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:41.284223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:41.284639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:41.414309 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:41.421863 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-201aa594-9711-4bc0-b783-99d3f893d424 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1680 microversion: 2.1 time: 0.150975 Oct 12 02:33:41.422418 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 325/651] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:41 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1680 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:42.444657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:42.445301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:42.453707 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:42.454188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:42.454617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:42.455303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:42.455753 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:42.456171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:42.584293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:42.592344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-61abcd8d-36b7-433a-b067-d192127ab875 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1680 microversion: 2.1 time: 0.151579 Oct 12 02:33:42.592941 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 327/652] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:42 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:43.616623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:43.617260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:43.623828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:43.624306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:43.624762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:43.625439 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:43.625887 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:43.626304 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:43.752245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:43.760140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-84ad4334-d6bf-42fd-8bcb-c68740e50d42 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1680 microversion: 2.1 time: 0.147307 Oct 12 02:33:43.760708 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 326/653] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:43 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1680 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:44.783613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:44.784220 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:44.786587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:44.792001 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:44.792491 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:44.792956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:44.794250 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:44.794705 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:44.795133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:44.953515 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:44.961609 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-85193807-a25d-45b0-953f-620ae6b582af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1680 microversion: 2.1 time: 0.181786 Oct 12 02:33:44.962181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 328/654] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:44 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1680 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:45.988284 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:45.989069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:45.996393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:45.997741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:46.002593 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:46.003560 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:46.004285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:46.004997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:46.076792 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:46.084212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:46.096762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:46.104142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:46.104741 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:46.105540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:46.106751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:46.107374 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:46.206728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:46.216754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-03c67597-4bd1-4022-9f6f-816c48927f07 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1612 microversion: 2.1 time: 0.232800 Oct 12 02:33:46.218506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 327/655] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:45 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1612 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:46.248533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "459cc4e1-2d7e-451d-b4d6-c7d0753355fd"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:46.261213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:46.287874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:46.288385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:46.288859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:46.291573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:46.292046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:46.292469 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:46.614060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:46.623862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f852f80a-889c-4d6c-ba23-43b071e42878 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1593 microversion: 2.1 time: 0.550532 Oct 12 02:33:46.625251 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 329/656] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:46 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1593 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:33:46.629943 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-41b8f018-061f-4778-a7bb-cb3f4afa7329 req-37f225e8-8029-4f2a-8dcb-ba98ebe54a71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dc118ab-0194-4cbb-b335-caf4c25cca7e", "name": "network-vif-deleted", "tag": "79216b85-6ffe-4aa2-b822-11695d819ed3"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:46.640319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-41b8f018-061f-4778-a7bb-cb3f4afa7329 req-37f225e8-8029-4f2a-8dcb-ba98ebe54a71 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:46.641344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-41b8f018-061f-4778-a7bb-cb3f4afa7329 req-37f225e8-8029-4f2a-8dcb-ba98ebe54a71 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:46.642008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-41b8f018-061f-4778-a7bb-cb3f4afa7329 req-37f225e8-8029-4f2a-8dcb-ba98ebe54a71 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:46.656307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-41b8f018-061f-4778-a7bb-cb3f4afa7329 req-37f225e8-8029-4f2a-8dcb-ba98ebe54a71 service nova] Creating event network-vif-deleted:79216b85-6ffe-4aa2-b822-11695d819ed3 for instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e on hv-cinder-84914 Oct 12 02:33:46.666166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-41b8f018-061f-4778-a7bb-cb3f4afa7329 req-37f225e8-8029-4f2a-8dcb-ba98ebe54a71 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038156 Oct 12 02:33:46.667487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 330/657] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:46.845305 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2a973906-7d3c-4959-af4a-0c446bf52baa tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.604146 Oct 12 02:33:46.846504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 328/658] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:33:46 2022] POST /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments => generated 194 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:47.243011 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:47.244298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:47.251796 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:47.252526 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:47.253163 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:47.254031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:47.254641 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:47.255221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:47.293488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:47.294225 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:47.294816 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:47.295451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'fault' on Instance uuid 7dc118ab-0194-4cbb-b335-caf4c25cca7e {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:33:47.351948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:47.368100 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6b2ab58c-d9a3-42eb-b85f-c8ff01ed4209 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 200 len: 1330 microversion: 2.1 time: 0.129527 Oct 12 02:33:47.369927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 331/659] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:47 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 1330 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:48.394835 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:48.395553 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:48.403683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:48.404167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:48.405070 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:48.405297 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:48.405741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:48.406157 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:48.419951 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e HTTP/1.1 Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:33:48.421639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:33:48.423212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '7dc118ab-0194-4cbb-b335-caf4c25cca7e'} Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:33:48.424278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:33:48.425731 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:33:48.427259 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:33:48.427259 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-26e5c308-6c28-409d-9468-ce5dd9612ce2 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 404 len: 111 microversion: 2.1 time: 0.033173 Oct 12 02:33:48.432470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-05c55eda-6570-47ad-a669-fff95b6458b1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '.version_select at 0x7f59d345cca0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:48.437477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 329/660] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:48 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:33:48.454580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-05c55eda-6570-47ad-a669-fff95b6458b1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-61575296" status: 202 len: 0 microversion: 2.1 time: 0.026242 Oct 12 02:33:48.455165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 332/661] 10.222.0.23 () {58 vars in 1215 bytes} [Wed Oct 12 02:33:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-61575296 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:33:48.942300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-51d8a14c-eea9-4512-97d1-460462704cfc req-f4d0a42a-7866-4d0e-b88e-e4a352f4e504 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bacd4ab7-51ad-4943-9d16-92bd1fc1adfa", "tag": "299443a4-87d4-4257-8710-d074d71d9851"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:48.962266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-51d8a14c-eea9-4512-97d1-460462704cfc req-f4d0a42a-7866-4d0e-b88e-e4a352f4e504 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:48.963022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-51d8a14c-eea9-4512-97d1-460462704cfc req-f4d0a42a-7866-4d0e-b88e-e4a352f4e504 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:48.963571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-51d8a14c-eea9-4512-97d1-460462704cfc req-f4d0a42a-7866-4d0e-b88e-e4a352f4e504 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:48.979302 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-51d8a14c-eea9-4512-97d1-460462704cfc req-f4d0a42a-7866-4d0e-b88e-e4a352f4e504 service nova] Creating event network-changed:299443a4-87d4-4257-8710-d074d71d9851 for instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa on hv-cinder-84914 Oct 12 02:33:48.990105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-51d8a14c-eea9-4512-97d1-460462704cfc req-f4d0a42a-7866-4d0e-b88e-e4a352f4e504 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053144 Oct 12 02:33:48.991384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 330/662] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:49.684460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:49.685040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:49.691961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:49.693329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:49.693971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:49.694676 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:49.695131 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:49.695557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:49.783356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'services' on Instance uuid 76667a6b-65ab-4a9d-8af3-0440b0dad579 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:33:49.862426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cd2321d7-3f6d-4fb5-9f07-8bd9e330d78b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments/960f293c-f2d1-48a5-8f68-2c9c685f2c9a" status: 202 len: 0 microversion: 2.1 time: 0.181755 Oct 12 02:33:49.863402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 333/663] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:33:49 2022] DELETE /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments/960f293c-f2d1-48a5-8f68-2c9c685f2c9a => generated 0 bytes in 184 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:33:49.872220 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:49.875145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:49.882297 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:49.883115 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:49.883840 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:49.884790 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:49.885460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:49.886094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:49.927337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-eb192cba-b95b-4805-8e6f-1df2b5e03011 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059248 Oct 12 02:33:49.932635 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 331/664] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:33:49 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:50.268798 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-514024739"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:50.269440 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:50.275517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:50.275995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:50.276422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:50.277151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:50.277628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:50.278047 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:50.412749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.network.security_group_api [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Adding security group 6a67273a-7bab-41b0-a78f-cb2f6a061cad to port 299443a4-87d4-4257-8710-d074d71d9851 Oct 12 02:33:50.721225 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f8f55448-7bcf-4fdc-99e4-81d2a2f4666a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/action" status: 202 len: 0 microversion: 2.1 time: 0.456972 Oct 12 02:33:50.724759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 334/665] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:33:50 2022] POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/action => generated 0 bytes in 459 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:33:50.729843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:50.730944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:50.737114 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:50.737595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:50.738026 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:50.738711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:50.739158 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:50.739573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:50.892573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:50.901373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7a5e7dce-871f-4d55-825c-9c87933652c5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1757 microversion: 2.1 time: 0.175356 Oct 12 02:33:50.901931 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 332/666] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:50 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1757 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:33:50.940095 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:50.942307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:50.948259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:50.948771 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:50.949212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:50.950479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:50.950935 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:50.951355 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:50.980392 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4ac34e5c-5901-40f5-ad93-5cc2cacf2eec tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044310 Oct 12 02:33:50.980982 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 335/667] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:33:50 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:51.011802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-e873b5f9-c27e-43ae-8f4c-dd5438deeea8 req-eccefd83-c485-427d-a540-7a371cf59105 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bacd4ab7-51ad-4943-9d16-92bd1fc1adfa", "tag": "299443a4-87d4-4257-8710-d074d71d9851"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:51.021513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-e873b5f9-c27e-43ae-8f4c-dd5438deeea8 req-eccefd83-c485-427d-a540-7a371cf59105 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:51.022090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-e873b5f9-c27e-43ae-8f4c-dd5438deeea8 req-eccefd83-c485-427d-a540-7a371cf59105 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:51.022522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-e873b5f9-c27e-43ae-8f4c-dd5438deeea8 req-eccefd83-c485-427d-a540-7a371cf59105 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:51.034272 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-e873b5f9-c27e-43ae-8f4c-dd5438deeea8 req-eccefd83-c485-427d-a540-7a371cf59105 service nova] Creating event network-changed:299443a4-87d4-4257-8710-d074d71d9851 for instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa on hv-cinder-84914 Oct 12 02:33:51.039145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-e873b5f9-c27e-43ae-8f4c-dd5438deeea8 req-eccefd83-c485-427d-a540-7a371cf59105 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031057 Oct 12 02:33:51.039937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 333/668] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:51.993170 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:51.995273 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:52.001601 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:52.002076 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:52.002501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:52.003175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:52.003622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:52.004039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:52.033217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3128248b-4d31-4651-a560-b8118a27454f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043988 Oct 12 02:33:52.033759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 336/669] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:33:51 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:53.047261 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:53.049403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:53.055454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.055934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.056366 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.057087 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.057541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.057960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.085983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b997a86a-17b5-4636-9b9e-57372471f88d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044189 Oct 12 02:33:53.086521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 334/670] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:33:53 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:53.547440 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:53.547934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:53.554870 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.555359 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.555821 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.556659 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.557142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.557439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.587040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bc68d6f5-c3f3-436b-a144-3950dc034fe2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments/02b78871-902d-4683-8a9a-c874f7431d50" status: 200 len: 194 microversion: 2.20 time: 0.046182 Oct 12 02:33:53.587670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 337/671] 10.222.0.23 () {60 vars in 1454 bytes} [Wed Oct 12 02:33:53 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments/02b78871-902d-4683-8a9a-c874f7431d50 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:33:53.656411 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:53.656868 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:53.663316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.663805 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.664239 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.664960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.665413 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.665845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.718873 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:53.793969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lazy-loading 'services' on Instance uuid 48eb4bd4-2680-4fc6-85c0-dae85fb740fa {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:33:53.908320 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e7e46cae-5cad-44b9-b995-b27bf3c97784 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments/02b78871-902d-4683-8a9a-c874f7431d50" status: 202 len: 0 microversion: 2.20 time: 0.256703 Oct 12 02:33:53.908880 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 335/672] 10.222.0.23 () {60 vars in 1457 bytes} [Wed Oct 12 02:33:53 2022] DELETE /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments/02b78871-902d-4683-8a9a-c874f7431d50 => generated 0 bytes in 258 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:33:53.917479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:53.919613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:53.925496 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.925977 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.926406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.927092 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:53.927539 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:53.927965 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:53.959424 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b8c8a89d-7962-4dff-9d3b-b4618431dfa7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045826 Oct 12 02:33:53.964360 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 338/673] 10.222.0.23 () {60 vars in 1343 bytes} [Wed Oct 12 02:33:53 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:33:54.099490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:54.101774 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:54.109442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.109948 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.110637 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:54.111329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.111787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.112208 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:54.141596 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9d5adfeb-73aa-4dbf-aac6-0213d3f6f5fc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046473 Oct 12 02:33:54.142152 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 336/674] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:33:54 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:33:54.562371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:54.564582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:54.570860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.571341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.571773 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:54.572450 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.572927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.573346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:54.596256 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lazy-loading 'flavor' on Instance uuid ac5527bc-4e21-4157-9415-41e66105dc9c {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:33:54.719739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5b77f3aa-b7f5-45d1-b7ec-03323b5ce39c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/action" status: 200 len: 32 microversion: 2.1 time: 0.161010 Oct 12 02:33:54.724435 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 339/675] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:33:54 2022] POST /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/action => generated 32 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:33:54.729441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:54.730052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:54.737808 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.738291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.738723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:54.739413 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.739866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.740840 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:54.881470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:54.890035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b0726200-4f83-4936-b4e2-744c0acf909e tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.164364 Oct 12 02:33:54.890602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 337/676] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:54 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:54.972599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:54.974814 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:54.980947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.981403 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.981827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:54.982498 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:54.982942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:54.983355 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.016280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-388498f2-ee88-40d4-b6ec-28f27b28a3d3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047943 Oct 12 02:33:55.016862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 340/677] 10.222.0.23 () {60 vars in 1343 bytes} [Wed Oct 12 02:33:54 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:33:55.155217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:55.157454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:55.164902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.165395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.166733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.167434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.168085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.168673 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.203744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c2d047a5-de18-4144-8e00-bb3a9fda41a6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053483 Oct 12 02:33:55.204310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 338/678] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:33:55 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:33:55.270779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:55.271285 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:55.277804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.278289 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.278719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.279415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.279868 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.280284 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.305134 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:33:55.333570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:33:55.398088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ff43ac35-f820-4004-b833-0dfb9ab95cab tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 204 len: 0 microversion: 2.1 time: 0.128084 Oct 12 02:33:55.399395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 341/679] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:33:55 2022] DELETE /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:33:55.408172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:55.408923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:55.415131 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.415719 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.416221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.417043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.417562 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.418136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.669207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:55.676784 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d4758119-f320-43da-acbb-3d9d684c28b8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1681 microversion: 2.1 time: 0.272495 Oct 12 02:33:55.677359 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 339/680] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:55 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1681 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:55.915018 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:55.915739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:55.922697 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.923119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.923554 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:55.924243 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:55.924718 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:55.925147 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:56.029646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:56.031981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:56.037964 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:56.038545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:56.039102 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:56.039863 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:56.040449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:56.041078 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:56.071877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-591f4b68-4f20-439a-ab9e-c3126ab54671 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047113 Oct 12 02:33:56.072531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 340/681] 10.222.0.23 () {60 vars in 1343 bytes} [Wed Oct 12 02:33:56 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 12 02:33:56.131958 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:56.140238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2cc6b351-2116-41f2-bfc8-e7001322e6d2 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.229464 Oct 12 02:33:56.140946 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 342/682] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:55 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:56.700463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:56.701116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:56.707200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:56.707677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:56.708107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:56.708846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:56.709300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:56.709713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:56.867620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:56.878582 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a3646055-5122-4eda-803e-0780f65f82bf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1681 microversion: 2.1 time: 0.182408 Oct 12 02:33:56.879177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 341/683] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:56 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1681 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:57.083980 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:57.086303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:57.093346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.093843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.094279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.094992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.095449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.095912 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.130682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6c6d4990-5386-4780-9b55-72cf7660a141 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.050741 Oct 12 02:33:57.131253 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 343/684] 10.222.0.23 () {60 vars in 1343 bytes} [Wed Oct 12 02:33:57 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:33:57.165530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:57.166821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:57.175271 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.175763 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.176198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.176940 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.177419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.177838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.214634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:57.215380 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:57.226455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.227088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.227518 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.228364 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.228845 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.229266 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.255607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:33:57.364269 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d0966a08-6cbb-4895-bda9-184386e17063 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 204 len: 0 microversion: 2.20 time: 0.154934 Oct 12 02:33:57.366710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 344/685] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 12 02:33:57 2022] DELETE /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 12 02:33:57.377054 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:57.377984 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:57.389273 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.389796 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.390395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.391416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.391876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.392371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.442494 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:57.450945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-77af93ff-6625-434c-a90b-fa4472dd78c0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.289963 Oct 12 02:33:57.451594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 342/686] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:57 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:57.623375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:57.631850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:57.636108 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:57.636108 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-09cea613-43d7-4962-a0c5-1909f386d96d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.263925 Oct 12 02:33:57.636108 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 345/687] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:33:57 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:33:57.904014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:57.904639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:57.911515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.912035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.912658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.913408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:57.914066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:57.914497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:57.978578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:57.989635 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d5d2c768-10ba-40cd-98c4-5f78c79a9b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1613 microversion: 2.1 time: 0.090119 Oct 12 02:33:57.997013 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 343/688] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:57 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1613 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:58.474240 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:58.474850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:58.481547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:58.482030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:58.482635 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:58.483535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:58.483979 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:58.484401 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:58.504289 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-4e54dba7-ea48-492e-8631-d1198df724f1 req-f868f26e-a080-4286-a793-a7d2ad7ea0fa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76667a6b-65ab-4a9d-8af3-0440b0dad579", "name": "network-vif-deleted", "tag": "a0229b37-24b4-4585-84cc-d3276e3660d3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:33:58.514962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-4e54dba7-ea48-492e-8631-d1198df724f1 req-f868f26e-a080-4286-a793-a7d2ad7ea0fa service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:58.515700 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-4e54dba7-ea48-492e-8631-d1198df724f1 req-f868f26e-a080-4286-a793-a7d2ad7ea0fa service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:58.516132 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-4e54dba7-ea48-492e-8631-d1198df724f1 req-f868f26e-a080-4286-a793-a7d2ad7ea0fa service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:58.527222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-4e54dba7-ea48-492e-8631-d1198df724f1 req-f868f26e-a080-4286-a793-a7d2ad7ea0fa service nova] Creating event network-vif-deleted:a0229b37-24b4-4585-84cc-d3276e3660d3 for instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 on hv-cinder-84914 Oct 12 02:33:58.532641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-4e54dba7-ea48-492e-8631-d1198df724f1 req-f868f26e-a080-4286-a793-a7d2ad7ea0fa service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032716 Oct 12 02:33:58.533331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 344/689] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:33:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:33:58.620079 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:58.629966 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-37c3f633-d793-47b3-a98e-7a55007ec7c9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.159635 Oct 12 02:33:58.630730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 346/690] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:58 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:58.661043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:58.661751 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:58.668960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:58.669443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:58.669922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:58.670604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:58.671055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:58.671476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:58.802358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:58.812198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:58.813702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:58.814855 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2f25650e-b53b-4442-a8ec-14c3a35d03ef tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.158204 Oct 12 02:33:58.815620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 345/691] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:33:58 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:33:59.015109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:59.016082 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:59.023404 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:59.023938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:59.024369 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:59.025188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:59.025636 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:59.026093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:59.092487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:59.102182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9f6c13cb-fc26-42c6-b850-7459b4a2680a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1325 microversion: 2.1 time: 0.091418 Oct 12 02:33:59.103202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 347/692] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:59 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1325 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:59.654921 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:59.655627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:59.662105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:59.662611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:59.663236 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:59.663984 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:59.664422 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:59.665026 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:59.840194 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:33:59.840751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:33:59.847164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:59.847638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:59.848071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:59.848784 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:33:59.849238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:33:59.849658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:33:59.940909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:59.950513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6dae8a04-7735-46f7-b13a-ad9840dba948 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.300204 Oct 12 02:33:59.951438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 346/693] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:33:59 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:33:59.987057 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:59.995015 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:59.997051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:33:59.998479 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-167da5cd-724a-4695-ae47-eca523043949 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1723 microversion: 2.20 time: 0.162288 Oct 12 02:33:59.999040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 348/694] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:33:59 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:34:00.130484 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:00.130896 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:00.137832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:00.138315 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:00.138922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:00.139609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:00.140057 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:00.140471 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:00.171473 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:00.171942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:00.172365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:00.172830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'fault' on Instance uuid 76667a6b-65ab-4a9d-8af3-0440b0dad579 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:34:00.240672 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:00.248962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-04192b3c-c6dd-4ecc-9c0e-c7f38bc2fa96 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1330 microversion: 2.1 time: 0.122412 Oct 12 02:34:00.249499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 347/695] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:00 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1330 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:00.407777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-a03b61ee-73fc-45db-bd9f-9568658fce11 req-bf1ef500-47ed-4bd7-8caa-071d05999a44 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48eb4bd4-2680-4fc6-85c0-dae85fb740fa", "name": "network-vif-deleted", "tag": "0a31a866-f72c-4f7d-822f-7d54a4e3afec"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:00.417194 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-a03b61ee-73fc-45db-bd9f-9568658fce11 req-bf1ef500-47ed-4bd7-8caa-071d05999a44 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:00.418240 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-a03b61ee-73fc-45db-bd9f-9568658fce11 req-bf1ef500-47ed-4bd7-8caa-071d05999a44 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:00.418660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-a03b61ee-73fc-45db-bd9f-9568658fce11 req-bf1ef500-47ed-4bd7-8caa-071d05999a44 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:00.429731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-a03b61ee-73fc-45db-bd9f-9568658fce11 req-bf1ef500-47ed-4bd7-8caa-071d05999a44 service nova] Creating event network-vif-deleted:0a31a866-f72c-4f7d-822f-7d54a4e3afec for instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa on hv-cinder-84914 Oct 12 02:34:00.435573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-a03b61ee-73fc-45db-bd9f-9568658fce11 req-bf1ef500-47ed-4bd7-8caa-071d05999a44 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031700 Oct 12 02:34:00.436486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 349/696] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:00.974378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:00.975073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:00.981498 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:00.982067 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:00.982560 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:00.983345 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:00.983856 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:00.984335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:01.023855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:01.035804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:01.043105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:01.043857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:01.044292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:01.045011 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:01.045671 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:01.046094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:01.110995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:01.121899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:01.123570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:01.124994 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-996cc75e-1c5a-4b31-b5ce-12dbfd4e5846 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1655 microversion: 2.20 time: 0.105047 Oct 12 02:34:01.135568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:01.138078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 350/697] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:01 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1655 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:34:01.144164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d9c33768-6725-495e-bfa4-069ce09003f0 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.173751 Oct 12 02:34:01.144687 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 348/698] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:00 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:01.271610 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:01.272340 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:01.278350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:01.278833 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:01.279257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:01.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:01.280665 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:01.281008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:01.318429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:01.318917 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:01.319345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:01.319788 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'fault' on Instance uuid 76667a6b-65ab-4a9d-8af3-0440b0dad579 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:34:01.361439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:01.369369 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f9603195-6058-4a63-973a-d733302bf68b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1330 microversion: 2.1 time: 0.101706 Oct 12 02:34:01.370031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 351/699] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:01 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1330 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:02.151445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:02.152555 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:02.158915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:02.159392 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:02.159821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:02.160496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:02.160970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:02.161386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:02.166989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:02.168280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:02.174560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:02.175036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:02.175460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:02.176142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:02.176590 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:02.177027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:02.221757 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:02.229701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:02.231717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:02.232898 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bf08e408-87ec-4190-9c92-87d5cd401311 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1655 microversion: 2.20 time: 0.085288 Oct 12 02:34:02.233453 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 349/700] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:02 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1655 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:02.308952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:02.317336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b1a31d62-7a79-42b6-8b8d-8d007272ba95 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.154041 Oct 12 02:34:02.317901 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 352/701] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:02 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:02.392793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:02.393384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:02.399188 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:02.399663 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:02.400092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:02.400799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:02.401244 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:02.401659 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:02.432630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:02.433153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:02.433577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:02.434016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'fault' on Instance uuid 76667a6b-65ab-4a9d-8af3-0440b0dad579 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:34:02.477317 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:02.485183 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-082350d5-727d-4016-ba1a-a848b00e7f8d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 200 len: 1330 microversion: 2.1 time: 0.095953 Oct 12 02:34:02.485742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 350/702] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:02 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 1330 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:03.258639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:03.259259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:03.265732 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:03.266223 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:03.266658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:03.267350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:03.267811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:03.268236 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:03.298596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:03.299006 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:03.299438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:03.299885 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lazy-loading 'fault' on Instance uuid 48eb4bd4-2680-4fc6-85c0-dae85fb740fa {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:34:03.339825 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:03.341038 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:03.346555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:03.349048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:03.349563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:03.350278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:03.352021 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:03.352604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:03.353145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:03.355918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:03.358126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:03.361297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3e6ba76b-8fe0-4327-aa79-537f55942284 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 200 len: 1372 microversion: 2.20 time: 0.106599 Oct 12 02:34:03.362388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 353/703] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:03 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:03.371247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:03.491533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:03.500297 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a528ca5f-6b47-415b-b0c0-6d4f00ea529a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.164427 Oct 12 02:34:03.500958 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 351/704] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:03 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:03.507270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:03.507895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:03.513714 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:03.514195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:03.514626 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:03.515312 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:03.515759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:03.516178 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:03.529600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 HTTP/1.1 Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:34:03.531205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:34:03.532843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '76667a6b-65ab-4a9d-8af3-0440b0dad579'} Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:34:03.533808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:34:03.535262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:03.536596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:34:03.536596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c3ec4d77-9ee4-4a5f-abda-66a07ada851a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 404 len: 111 microversion: 2.1 time: 0.029186 Oct 12 02:34:03.536596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 354/705] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:03 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:34:03.541746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4663b62a-75a6-4cf2-a2c0-e08ddfeaafdf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '.version_select at 0x7f59d39e0dc0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:03.561509 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4663b62a-75a6-4cf2-a2c0-e08ddfeaafdf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-780906542" status: 202 len: 0 microversion: 2.1 time: 0.023447 Oct 12 02:34:03.561997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 352/706] 10.222.0.23 () {58 vars in 1218 bytes} [Wed Oct 12 02:34:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-780906542 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:34:04.193732 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6067a2d3-4c51-411d-906d-aca2fdaa9968 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: .version_select at 0x7f59d3509700>, body: {"keypair": {"name": "tempest-keypair-270069979", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB3nlbJOJA1XOKNlCvMleYgb4M1pUSP0rWT4jQ+kYF5cbiYVNvLWy7gKAsM9wIM7quGnntoshR00qrsVkDPifcMY6Eleyoz2XjS530F+PtsE6g+6oeWoBmCyT6troGlIeA=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:04.199699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=83,nova_api:DELETE=1,nova_api:UPDATE=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:04.212682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-6067a2d3-4c51-411d-906d-aca2fdaa9968 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:34:04.217116 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-6067a2d3-4c51-411d-906d-aca2fdaa9968 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:34:04.238286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6067a2d3-4c51-411d-906d-aca2fdaa9968 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.048398 Oct 12 02:34:04.238855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 355/707] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:34:04 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:04.389970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:04.390629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:04.396423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:04.396958 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:04.397396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:04.399094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:04.399554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:04.399978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:04.413477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] HTTP exception thrown: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Request method failure captured: Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa HTTP/1.1 Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaLTZf3hgmHzlK_C5LRX3NkyBEyTPu97mBszC48ey8TfKxHZ5Xwm1OSjbhNRDfu6az55cMVbT2b1UQJJRTOVbmicV1cVFK4d0Ia2uGNL8vRrUrQX61jMMFFyDtcjLIrxf5lvkeSXV-zHLvO6WZ4QYh1jwpdmF4zw11AKgY1DoAyrIo Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Openstack-Nova-Api-Version: 2.20 Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:34:04.415127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_7f6eb3e59c464c749edb69e8370f54e5", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 43f16ca13925411fb76a330b6c95395d Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:34:04.416641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '48eb4bd4-2680-4fc6-85c0-dae85fb740fa'} Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:34:04.420944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:34:04.422382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:34:04.423615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Returning 404 to user: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:34:04.423615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7040783a-38ad-4b08-964f-d9a5f799561c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 404 len: 111 microversion: 2.20 time: 0.031696 Oct 12 02:34:04.423615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 353/708] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:04 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:34:04.426202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-dfe03776-69d4-48d4-9054-9039eb35bec2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '.version_select at 0x7f59d33bef70>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:04.448072 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-dfe03776-69d4-48d4-9054-9039eb35bec2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2139911655" status: 204 len: 0 microversion: 2.20 time: 0.025985 Oct 12 02:34:04.448468 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 356/709] 10.222.0.23 () {60 vars in 1262 bytes} [Wed Oct 12 02:34:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2139911655 => generated 0 bytes in 27 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 12 02:34:04.523838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:04.524491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:04.531879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:04.532091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:04.532664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:04.533429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:04.534147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:04.534466 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:04.716526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:04.727398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3fa7bad1-321c-46fa-a57d-07077738d4b7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.207742 Oct 12 02:34:04.727971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 354/710] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:04 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:05.522154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c8d63865-fdd5-4e48-a4bd-1faea4465226 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'create', calling method: .version_select at 0x7f59d33cd160>, body: {"keypair": {"name": "tempest-keypair-478946019", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHvudnrAOs5q6fSmf7PT314bfjcX2CgRIfmE2eP/wSF5CkXKAPRY//flJv8gS0A4PIOCzuJGPZuTKYaptbZWRYR78G0rMVSL5eQ6bpQYvHoKV4nVxjDRATyBZDe6BYuKww=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:05.537687 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-c8d63865-fdd5-4e48-a4bd-1faea4465226 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:34:05.541383 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-c8d63865-fdd5-4e48-a4bd-1faea4465226 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for user 43f16ca13925411fb76a330b6c95395d and project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:34:05.562304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c8d63865-fdd5-4e48-a4bd-1faea4465226 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.044674 Oct 12 02:34:05.562909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 357/711] 10.222.0.23 () {62 vars in 1200 bytes} [Wed Oct 12 02:34:05 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 46 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 12 02:34:05.753050 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:05.753670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:05.768386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:05.768905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:05.769337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:05.770112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:05.770565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:05.771042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:05.905480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:05.914542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-68d4c28e-eda5-4d59-a837-f7e968c0a250 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.165431 Oct 12 02:34:05.915543 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 355/712] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:05 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:06.048134 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-813132200", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "21fdf3f0-6b90-400a-b2fc-5e540c8c18e8"}], "security_groups": [{"name": "tempest-securitygroup--1028841812"}], "key_name": "tempest-keypair-270069979", "user_data": "}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:06.225525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] validate_networks() for [('21fdf3f0-6b90-400a-b2fc-5e540c8c18e8', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:34:06.893155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:34:06.893716 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:34:06.894028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:34:06.894425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:34:06.894878 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:34:06.904055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:06.904528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:06.904983 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:06.906029 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:06.906486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:06.906970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:06.926344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:34:06.931169 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:34:06.940155 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:06.961657 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:06.962916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:34:06.967242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:34:06.973775 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:06.974845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:06.975601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:06.976919 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:06.978898 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:06.980432 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:07.032361 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4e2126bf-6669-4f18-8a06-175ff1977401 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.989680 Oct 12 02:34:07.036951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 358/713] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:34:06 2022] POST /compute/v2.1/servers => generated 402 bytes in 994 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:34:07.047747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7c265f17-b9a0-4da0-810c-67ed91792877 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:07.048436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7c265f17-b9a0-4da0-810c-67ed91792877 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:07.192217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:07.202449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7b9e7e23-7c37-42f0-a1f4-35e0b9dfa8c3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.266356 Oct 12 02:34:07.204339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 356/714] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:06 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:07.208745 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7c265f17-b9a0-4da0-810c-67ed91792877 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:07.216594 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7c265f17-b9a0-4da0-810c-67ed91792877 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1240 microversion: 2.1 time: 0.172817 Oct 12 02:34:07.218232 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 359/715] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:07 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1240 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:07.537461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-627388443", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8e5a7b02-9fc0-4dda-b1f4-8267371dc191"}], "security_groups": [{"name": "tempest-securitygroup--181258085"}], "key_name": "tempest-keypair-478946019", "user_data": "}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:07.645285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] validate_networks() for [('8e5a7b02-9fc0-4dda-b1f4-8267371dc191', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:34:08.021849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:34:08.022511 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:34:08.022737 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:34:08.023132 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:34:08.023559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:34:08.032113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:08.032582 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:08.033034 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:08.033745 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:08.034219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:08.034634 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:08.050882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:34:08.054644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Getting quotas for user 43f16ca13925411fb76a330b6c95395d and project 7f6eb3e59c464c749edb69e8370f54e5. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:34:08.065764 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:34:08.068461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:34:08.115964 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fe7ff1b5-424b-4c3f-83ab-4ddcc1bad099 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.583003 Oct 12 02:34:08.116757 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 357/716] 10.222.0.23 () {62 vars in 1188 bytes} [Wed Oct 12 02:34:07 2022] POST /compute/v2.1/servers => generated 401 bytes in 584 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 12 02:34:08.128244 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3277d871-1cb1-497d-8e22-0cd5c86d4d8e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:08.128916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3277d871-1cb1-497d-8e22-0cd5c86d4d8e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:08.185647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3277d871-1cb1-497d-8e22-0cd5c86d4d8e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:08.193091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3277d871-1cb1-497d-8e22-0cd5c86d4d8e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:08.194660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3277d871-1cb1-497d-8e22-0cd5c86d4d8e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:08.195799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3277d871-1cb1-497d-8e22-0cd5c86d4d8e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1280 microversion: 2.20 time: 0.071449 Oct 12 02:34:08.202131 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 360/717] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:08 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1280 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:08.229574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:08.230215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:08.237897 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:08.238386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:08.239051 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:08.239754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:08.240206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:08.240829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:08.249003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:08.249611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:08.257445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:08.258083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:08.258543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:08.259323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:08.259776 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:08.260195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:08.340410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:08.348072 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-7cb7fa5d-73bf-406a-b1db-546f1591ce54 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1292 microversion: 2.1 time: 0.104259 Oct 12 02:34:08.357028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 361/718] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:08 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1292 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:08.533249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:08.541459 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4c6d3ac0-1050-4878-8169-34a3a41496fd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.315978 Oct 12 02:34:08.542482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 358/719] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:08 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:09.221356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:09.222612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:09.229104 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:09.229847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:09.230283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:09.230989 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:09.231443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:09.231865 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:09.236417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=96,nova_cell1:INSERT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:09.304264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:09.313750 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:09.315406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:09.316729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0ed3d9ed-6557-4ed9-9833-160a1e7da673 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1332 microversion: 2.20 time: 0.099279 Oct 12 02:34:09.317496 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 362/720] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:09 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1332 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:09.374014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:09.374619 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:09.377085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:09.382514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:09.383002 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:09.383432 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:09.384317 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:09.384798 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:09.385222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:09.578806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:09.581944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:09.601489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:09.602264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:09.602884 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:09.606626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:09.606856 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:09.607387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:09.608410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:09.616371 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-689b0d2d-78d6-42c4-b309-930733f83120 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.246272 Oct 12 02:34:09.617638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 359/721] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:09 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:09.801922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:09.812439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e1fe8555-8a91-4a19-811d-131868be956f tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.248902 Oct 12 02:34:09.813218 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 363/722] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:09 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:10.126982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-55555373-28be-430e-b450-4cbb47cfca12 req-bc782ede-d3fb-4c2c-a0f7-7c2fe791294d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "928a91d5-600b-4707-9428-2e716f411885", "tag": "8ff097ef-11a1-4847-9e06-95f2c84807d0"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:10.137465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-55555373-28be-430e-b450-4cbb47cfca12 req-bc782ede-d3fb-4c2c-a0f7-7c2fe791294d service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:10.138186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-55555373-28be-430e-b450-4cbb47cfca12 req-bc782ede-d3fb-4c2c-a0f7-7c2fe791294d service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:10.138785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-55555373-28be-430e-b450-4cbb47cfca12 req-bc782ede-d3fb-4c2c-a0f7-7c2fe791294d service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:10.150129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-55555373-28be-430e-b450-4cbb47cfca12 req-bc782ede-d3fb-4c2c-a0f7-7c2fe791294d service nova] Creating event network-changed:8ff097ef-11a1-4847-9e06-95f2c84807d0 for instance 928a91d5-600b-4707-9428-2e716f411885 on hv-cinder-84914 Oct 12 02:34:10.154952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-55555373-28be-430e-b450-4cbb47cfca12 req-bc782ede-d3fb-4c2c-a0f7-7c2fe791294d service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031651 Oct 12 02:34:10.155561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 360/723] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:10.343506 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:10.344931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:10.352538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:10.353424 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:10.354032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:10.355858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:10.356488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:10.357105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:10.643092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:10.643947 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:10.650557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:10.651049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:10.652063 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:10.652803 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:10.653258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:10.653704 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:10.833851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:10.843747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:10.845398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:10.846633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fab09587-f373-49b3-b4a6-29a43550cc50 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.507695 Oct 12 02:34:10.847186 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 364/724] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:10 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:10.856144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:10.856988 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:10.863442 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:10.863928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:10.864353 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:10.865066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:10.865721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:10.866264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:10.884594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:10.895287 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b5afc891-6138-4369-abbd-6111fafb81e6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.256952 Oct 12 02:34:10.896633 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 361/725] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:10 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:11.120771 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:11.128999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0cf800b9-a8b8-4b3b-bcd0-c2514a6441ac tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.280536 Oct 12 02:34:11.129557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 365/726] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:10 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:11.515308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-b804043a-eee0-4f7a-8325-c773ca70f55d req-ffb00185-cb95-4a50-86a2-90bd1e2d8397 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a61f7314-595f-4a3e-9288-0d25df1b9607", "tag": "fe5a040d-6b9b-4b90-a3c9-76a1085a756e"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:11.524630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b804043a-eee0-4f7a-8325-c773ca70f55d req-ffb00185-cb95-4a50-86a2-90bd1e2d8397 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:11.525144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b804043a-eee0-4f7a-8325-c773ca70f55d req-ffb00185-cb95-4a50-86a2-90bd1e2d8397 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:11.525575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b804043a-eee0-4f7a-8325-c773ca70f55d req-ffb00185-cb95-4a50-86a2-90bd1e2d8397 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:11.536133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-b804043a-eee0-4f7a-8325-c773ca70f55d req-ffb00185-cb95-4a50-86a2-90bd1e2d8397 service nova] Creating event network-changed:fe5a040d-6b9b-4b90-a3c9-76a1085a756e for instance a61f7314-595f-4a3e-9288-0d25df1b9607 on hv-cinder-84914 Oct 12 02:34:11.540826 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-b804043a-eee0-4f7a-8325-c773ca70f55d req-ffb00185-cb95-4a50-86a2-90bd1e2d8397 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029072 Oct 12 02:34:11.541576 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 362/727] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:34:11.871646 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:11.872247 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:11.878677 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:11.879163 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:11.879597 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:11.880292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:11.880779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:11.881205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:11.920400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:11.921267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:11.927829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:11.928323 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:11.928834 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:11.929536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:11.929992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:11.930421 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:12.084667 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:12.094478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:12.096445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:12.097830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ec7edeac-3ba3-4250-a8dd-372c88d34de5 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.230105 Oct 12 02:34:12.113486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 366/728] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:11 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:12.153602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:12.154846 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:12.159999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:12.160481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:12.160944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:12.161637 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:12.162088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:12.162504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:12.171581 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:12.180137 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5ce4f63a-c49e-4fae-8f6a-7178341747af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.264907 Oct 12 02:34:12.181306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 363/729] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:11 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:12.391815 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:12.400356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-71807384-ce85-47a9-8111-7b7a44f2ca77 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.251330 Oct 12 02:34:12.401859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 367/730] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:12 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:13.124740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:13.125353 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:13.131717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:13.132205 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:13.132648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:13.133711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:13.134175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:13.134599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:13.203836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:13.204576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:13.210867 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:13.211411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:13.211904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:13.212650 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:13.213219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:13.214358 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:13.271784 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:13.280077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:13.281521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:13.282645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a3f20596-964e-43ac-b686-a04dda1f6d8f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.161832 Oct 12 02:34:13.283190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 364/731] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:13 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:13.350023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:13.357551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a3a3b3ac-f632-4c3b-93ec-91b4d2fcebba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.157566 Oct 12 02:34:13.358111 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 368/732] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:13 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:13.424264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:13.424942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:13.430650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:13.431122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:13.431832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:13.433023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:13.433467 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:13.433882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:13.558334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:13.566289 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-83b96da3-999f-45a1-b7e5-142e3cc72489 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.145744 Oct 12 02:34:13.566845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 365/733] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:13 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:14.307300 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:14.307934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:14.314373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:14.314855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:14.315287 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:14.315971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:14.316422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:14.316876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:14.381089 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:14.381632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:14.388337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:14.388840 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:14.389270 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:14.389954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:14.390404 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:14.390838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:14.444229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:14.453051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:14.454489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:14.455574 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a0b7ebc3-0789-4f1e-953f-4be891c61456 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.152227 Oct 12 02:34:14.456125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 369/734] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:14 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:14.522114 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:14.531676 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f14795ec-78fe-4236-8b33-21e294328c7c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.154446 Oct 12 02:34:14.532264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 366/735] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:14 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:14.590453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:14.591055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:14.596925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:14.597411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:14.597846 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:14.598536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:14.598986 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:14.599409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:14.734783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:14.743591 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-699ab3b7-4cf5-46c0-8590-e07801bc2fc9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.157409 Oct 12 02:34:14.744292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 370/736] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:14 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:14.834139 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-6550f6d0-219c-4e5f-abd6-f5e58da9a70c req-be65f387-46c2-46af-a5a4-87552d02435e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "928a91d5-600b-4707-9428-2e716f411885", "name": "network-vif-plugged", "status": "completed", "tag": "8ff097ef-11a1-4847-9e06-95f2c84807d0"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:14.844960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-6550f6d0-219c-4e5f-abd6-f5e58da9a70c req-be65f387-46c2-46af-a5a4-87552d02435e service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:14.845435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-6550f6d0-219c-4e5f-abd6-f5e58da9a70c req-be65f387-46c2-46af-a5a4-87552d02435e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:14.845859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-6550f6d0-219c-4e5f-abd6-f5e58da9a70c req-be65f387-46c2-46af-a5a4-87552d02435e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:14.856779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-6550f6d0-219c-4e5f-abd6-f5e58da9a70c req-be65f387-46c2-46af-a5a4-87552d02435e service nova] Creating event network-vif-plugged:8ff097ef-11a1-4847-9e06-95f2c84807d0 for instance 928a91d5-600b-4707-9428-2e716f411885 on hv-cinder-84914 Oct 12 02:34:14.861872 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-6550f6d0-219c-4e5f-abd6-f5e58da9a70c req-be65f387-46c2-46af-a5a4-87552d02435e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031432 Oct 12 02:34:14.862656 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 367/737] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:15.481579 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:15.482138 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:15.488646 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:15.489171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:15.489602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:15.490283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:15.490731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:15.491176 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:15.555457 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:15.556146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:15.562173 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:15.562653 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:15.563084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:15.563771 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:15.564219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:15.564638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:15.618671 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:15.626807 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:15.628233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:15.629366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-923a57b1-6497-42fb-a8d6-5932159ea3ec tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.152347 Oct 12 02:34:15.629926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 371/738] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:15 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:15.691812 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:15.699407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e4131329-d10c-48c3-bd48-494e6b3e1232 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.148556 Oct 12 02:34:15.699978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 368/739] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:15 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:15.773492 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:15.774010 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:15.780354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:15.780890 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:15.781328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:15.782022 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:15.782474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:15.782896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:15.921811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:15.932619 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-150bd449-e637-4359-ae24-139d138c84d1 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.162996 Oct 12 02:34:15.934098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 372/740] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:15 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:16.546744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:16.549030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:16.554940 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.555415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.555846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.556524 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.556998 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.557912 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.631935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1de559c4-cb82-444f-8c4d-186c4e9a978a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/action" status: 202 len: 0 microversion: 2.1 time: 0.088924 Oct 12 02:34:16.635654 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 369/741] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:34:16 2022] POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:34:16.640775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:16.641381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:16.647734 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.648320 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.649216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.649596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.650055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.650470 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.654888 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:16.655959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:16.662050 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.662565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.663003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.663697 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.664150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.664574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.800459 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:16.801827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:16.809927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1d3e3428-b35b-4621-b711-82f9fed25402 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1749 microversion: 2.1 time: 0.172962 Oct 12 02:34:16.810517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 373/742] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:16 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1749 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:16.811482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:16.813007 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:16.814514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2b13aa2e-6009-4f57-a6d5-713e480677ff tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.163830 Oct 12 02:34:16.815181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:16.815365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 370/743] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:16 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:34:16.815786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:16.821569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.822043 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.822486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.823179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.823630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.824045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.876333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-54b60c24-21bd-4b6c-b9d5-9617d4b8dfc5 req-35baa2c7-d7eb-4502-982b-3df41aa4815a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a61f7314-595f-4a3e-9288-0d25df1b9607", "name": "network-vif-plugged", "status": "completed", "tag": "fe5a040d-6b9b-4b90-a3c9-76a1085a756e"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:16.885836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-54b60c24-21bd-4b6c-b9d5-9617d4b8dfc5 req-35baa2c7-d7eb-4502-982b-3df41aa4815a service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.886301 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-54b60c24-21bd-4b6c-b9d5-9617d4b8dfc5 req-35baa2c7-d7eb-4502-982b-3df41aa4815a service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.886741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-54b60c24-21bd-4b6c-b9d5-9617d4b8dfc5 req-35baa2c7-d7eb-4502-982b-3df41aa4815a service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.899710 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-54b60c24-21bd-4b6c-b9d5-9617d4b8dfc5 req-35baa2c7-d7eb-4502-982b-3df41aa4815a service nova] Creating event network-vif-plugged:fe5a040d-6b9b-4b90-a3c9-76a1085a756e for instance a61f7314-595f-4a3e-9288-0d25df1b9607 on hv-cinder-84914 Oct 12 02:34:16.905992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-54b60c24-21bd-4b6c-b9d5-9617d4b8dfc5 req-35baa2c7-d7eb-4502-982b-3df41aa4815a service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032863 Oct 12 02:34:16.906591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 371/744] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:34:16.915279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:16.956554 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:16.957747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:16.958519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:16.964474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cd0842f2-bfde-403a-a3d6-e65985216443 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.152910 Oct 12 02:34:16.965323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 374/745] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:16 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:16.965979 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.966489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.966927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.967636 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:16.968216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:16.968643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:16.974173 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:17.103068 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:17.112169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ce92a028-b59d-4117-98ce-27d914d88955 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1737 microversion: 2.1 time: 0.158576 Oct 12 02:34:17.112754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 372/746] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:16 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:17.841298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:17.841804 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:17.842198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:17.842446 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:17.848270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:17.848645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:17.848779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:17.849153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:17.849254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:17.849583 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:17.849889 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:17.850258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:17.850356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:17.850710 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:17.850850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:17.851130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:17.979589 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:17.988038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:17.989508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:17.990605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b7f9a4cc-92b2-4405-99d3-6e71ee088e72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.153281 Oct 12 02:34:17.991165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 375/747] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:17 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:17.994244 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:17.996037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:17.996653 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:18.002254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.002741 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.003169 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.003847 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-11560f45-e54e-4d91-adff-3fd5bc81db54 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1749 microversion: 2.1 time: 0.165751 Oct 12 02:34:18.003973 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.004299 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.007879 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.008018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 373/748] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:17 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1749 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:18.134247 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:18.135168 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:18.135816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:18.137936 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:18.138625 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=6 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:18.142531 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6d6c1f33-923c-4eb5-b7f4-b558368c157a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1366 microversion: 2.1 time: 0.150410 Oct 12 02:34:18.143184 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 376/749] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:17 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:18.144403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.144918 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.145379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.146075 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.146529 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.146947 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.282314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:18.290511 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-596c52df-2293-41d5-ad27-69852b347156 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1717 microversion: 2.1 time: 0.159095 Oct 12 02:34:18.291085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 374/750] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:18 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1717 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:18.408511 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:18.409026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:18.410870 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=18 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:18.416471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.417530 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.417970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.418669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.419119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.419542 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.500541 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] HTTP exception thrown: Cannot 'detach_volume' instance ac5527bc-4e21-4157-9415-41e66105dc9c while it is in vm_state rescued Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Request method failure captured: Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments/459cc4e1-2d7e-451d-b4d6-c7d0753355fd HTTP/1.1 Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaPYwNdGPkaZC6cHEUGZ0SkhncrUI5nVnVZxOFbWJ3oJ3B77igC0roWc-nMKs5IUYGUtOa_YaPxwFn9fOjAYUswTJuudkNK6KicjYPQGN35ZLr1BK_R3Kt1jRPqGECcut8i4CjPKiklVRegxUQM3z5ZDQ5sJ8YSuV1B4cub9phJoCc Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:34:18.502164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bd41b04d606949edbc9a3a2f88f991b2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: bcd7f1121dab4e8db4c747c51ed49240 Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:34:18.503513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Cannot 'detach_volume' instance ac5527bc-4e21-4157-9415-41e66105dc9c while it is in vm_state rescued Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'server_id': 'ac5527bc-4e21-4157-9415-41e66105dc9c', 'id': '459cc4e1-2d7e-451d-b4d6-c7d0753355fd'} Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance ac5527bc-4e21-4157-9415-41e66105dc9c in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:34:18.504208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance ac5527bc-4e21-4157-9415-41e66105dc9c while it is in vm_state rescued Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance ac5527bc-4e21-4157-9415-41e66105dc9c while it is in vm_state rescued Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:34:18.505683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Returning 409 to user: Cannot 'detach_volume' instance ac5527bc-4e21-4157-9415-41e66105dc9c while it is in vm_state rescued {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:34:18.506823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d6a4a9e3-d36f-4596-81dc-b1aad7fad665 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "DELETE /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments/459cc4e1-2d7e-451d-b4d6-c7d0753355fd" status: 409 len: 152 microversion: 2.1 time: 0.099022 Oct 12 02:34:18.506823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 377/751] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:34:18 2022] DELETE /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments/459cc4e1-2d7e-451d-b4d6-c7d0753355fd => generated 152 bytes in 100 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Oct 12 02:34:18.512429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:18.513134 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:18.520629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.521143 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.521574 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.522258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.522708 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.523156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.587910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b31d6f5e-6f70-4c86-b6b1-acfd53a70ddf tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "POST /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/action" status: 202 len: 0 microversion: 2.1 time: 0.079083 Oct 12 02:34:18.589482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 375/752] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:34:18 2022] POST /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/action => generated 0 bytes in 80 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:34:18.597395 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:18.598024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:18.603833 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.604304 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.605402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.606075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:18.606516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:18.606929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:18.736652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:18.740016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=100 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:18.745787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6315a06c-39e7-4ed4-b239-68bf1247bdef tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1725 microversion: 2.1 time: 0.152248 Oct 12 02:34:18.746348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 378/753] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:18 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:19.017140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:19.018030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:19.025006 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.025566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.026067 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.027409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.027915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.028654 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.029600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:19.030495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:19.040950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.042131 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.042677 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.043918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.044630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.045280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.238978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:19.248990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:19.252894 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:19.256886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:19.258172 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-22a44f61-6fd8-47c2-88ae-276201dda18a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1405 microversion: 2.20 time: 0.245847 Oct 12 02:34:19.258767 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 376/754] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:19 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1405 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:34:19.262491 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f1b5986b-f73c-4949-ad6b-382453f6481e tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1749 microversion: 2.1 time: 0.238983 Oct 12 02:34:19.263366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 379/755] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:19 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1749 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:19.265627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:19.266217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:19.272692 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.273425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.273928 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.274695 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.275235 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.275727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.407300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:19.414879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6bf73be0-1c76-44be-a3eb-753aa555495c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1556 microversion: 2.1 time: 0.154773 Oct 12 02:34:19.415438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 377/756] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:19 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1556 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:19.437370 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:19.437854 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:19.443643 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.444123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.444552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.445275 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.445725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.446142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.563490 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-731aac3c-5974-49e4-ab75-89368ae69240 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.129811 Oct 12 02:34:19.564220 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 380/757] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:34:19 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-interface => generated 284 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:19.768873 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:19.769526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:19.775475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.775957 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.776387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:19.777101 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:19.777552 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:19.777970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:20.187333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:20.195870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3c5cfdf0-d2a1-4fd9-b1d2-58afbc3b6c96 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1725 microversion: 2.1 time: 0.430959 Oct 12 02:34:20.196441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 378/758] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:19 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1725 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:20.286227 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:20.287496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:20.287629 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:20.289039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:20.296119 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:20.297084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:20.298500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:20.299426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:20.299969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:20.300103 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:20.300488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:20.301022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:20.301371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:20.301857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:20.302184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:20.302293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:20.451894 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:20.461189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:20.462710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:20.463929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-66131982-6aaf-4600-8656-8a7d8de71d0c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1596 microversion: 2.20 time: 0.182674 Oct 12 02:34:20.464505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 381/759] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:34:20 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1596 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:34:20.488903 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:20.489483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:20.495430 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:20.495984 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:20.496502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:20.497280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:20.497812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:20.498296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:20.578229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a8691109-c3a3-40e1-b321-d45a7882e96a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.093045 Oct 12 02:34:20.578922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 382/760] 10.222.0.23 () {60 vars in 1316 bytes} [Wed Oct 12 02:34:20 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 12 02:34:20.595871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:20.604056 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2b3912e3-49c1-47fe-9616-649ef7227dca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.322040 Oct 12 02:34:20.605405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 379/761] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:20 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:21.048742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-b5dd30da-a3f0-414e-9a33-87ece546a138 req-117e39d5-7b8b-4f1e-b87c-fa67c69b69f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "928a91d5-600b-4707-9428-2e716f411885", "tag": "8ff097ef-11a1-4847-9e06-95f2c84807d0"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:21.059260 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-b5dd30da-a3f0-414e-9a33-87ece546a138 req-117e39d5-7b8b-4f1e-b87c-fa67c69b69f3 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:21.059860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-b5dd30da-a3f0-414e-9a33-87ece546a138 req-117e39d5-7b8b-4f1e-b87c-fa67c69b69f3 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:21.060378 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-b5dd30da-a3f0-414e-9a33-87ece546a138 req-117e39d5-7b8b-4f1e-b87c-fa67c69b69f3 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:21.074530 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-b5dd30da-a3f0-414e-9a33-87ece546a138 req-117e39d5-7b8b-4f1e-b87c-fa67c69b69f3 service nova] Creating event network-changed:8ff097ef-11a1-4847-9e06-95f2c84807d0 for instance 928a91d5-600b-4707-9428-2e716f411885 on hv-cinder-84914 Oct 12 02:34:21.082396 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-b5dd30da-a3f0-414e-9a33-87ece546a138 req-117e39d5-7b8b-4f1e-b87c-fa67c69b69f3 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037365 Oct 12 02:34:21.083453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 383/762] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:21.219079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:21.219871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:21.227308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:21.227995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:21.228669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:21.229718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:21.230273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:21.231017 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:21.373532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:21.382311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-870b9c9a-0845-4099-b337-68a6a7ddff9c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1725 microversion: 2.1 time: 0.167318 Oct 12 02:34:21.382883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 380/763] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:21 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:21.629730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:21.630432 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:21.633182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=5,nova_api:DELETE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:21.639445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:21.639982 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:21.640415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:21.641212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:21.641645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:21.642058 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:21.858795 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-92472184-a36a-49e4-bd9a-8d9795187131 req-abbebc93-1072-4779-96ed-604c4f1f0b85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a61f7314-595f-4a3e-9288-0d25df1b9607", "tag": "fe5a040d-6b9b-4b90-a3c9-76a1085a756e"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:21.871830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-92472184-a36a-49e4-bd9a-8d9795187131 req-abbebc93-1072-4779-96ed-604c4f1f0b85 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:21.872860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-92472184-a36a-49e4-bd9a-8d9795187131 req-abbebc93-1072-4779-96ed-604c4f1f0b85 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:21.873540 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-92472184-a36a-49e4-bd9a-8d9795187131 req-abbebc93-1072-4779-96ed-604c4f1f0b85 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:21.887092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-92472184-a36a-49e4-bd9a-8d9795187131 req-abbebc93-1072-4779-96ed-604c4f1f0b85 service nova] Creating event network-changed:fe5a040d-6b9b-4b90-a3c9-76a1085a756e for instance a61f7314-595f-4a3e-9288-0d25df1b9607 on hv-cinder-84914 Oct 12 02:34:21.892820 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-92472184-a36a-49e4-bd9a-8d9795187131 req-abbebc93-1072-4779-96ed-604c4f1f0b85 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038390 Oct 12 02:34:21.893956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 381/764] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:21.921785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:21.930454 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cddf0671-1ee3-499a-b358-b78f252d6597 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.304878 Oct 12 02:34:21.931947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 384/765] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:21 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:22.408065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:22.408786 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:22.414881 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:22.415368 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:22.415800 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:22.416500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:22.417548 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:22.417974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:22.619403 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:22.632120 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-75e57b8f-ab10-4702-981e-8694b8cde72a tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1725 microversion: 2.1 time: 0.228725 Oct 12 02:34:22.633351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 382/766] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:22 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1725 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:22.954783 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:22.955423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:22.961858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:22.962344 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:22.962779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:22.963501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:22.965158 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:22.965589 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:23.103457 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:23.112242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c50082d4-08ca-4c28-8234-8e6891424015 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.161238 Oct 12 02:34:23.112828 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 385/767] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:22 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:23.655377 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:23.655977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:23.662279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:23.662811 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:23.663248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:23.663936 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:23.664390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:23.664838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:23.792496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:23.800499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4b5226d8-9acc-4afd-89d5-1d82e3d9e396 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1731 microversion: 2.1 time: 0.149020 Oct 12 02:34:23.801117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 383/768] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:23 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1731 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:23.823310 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:23.823797 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:23.829788 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:23.830206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:23.830633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:23.831362 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:23.831813 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:23.832230 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:24.060911 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ee9a8dad-f742-40ab-9c8d-3b9c6cb556de tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.241382 Oct 12 02:34:24.061526 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 386/769] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:34:23 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-interface => generated 285 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:24.136946 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:24.137551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:24.143445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:24.143929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:24.144359 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:24.145071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:24.145525 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:24.145941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:24.288609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:24.297680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2465d6a0-842f-48f5-aaf8-f1b1d1005f2b tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.165076 Oct 12 02:34:24.298518 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 384/770] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:24 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:34:24.909992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-d7d1b269-3dee-468f-8ccc-19cc352487ad req-38f6c57f-baa1-42b3-8e30-fad58c860e11 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac5527bc-4e21-4157-9415-41e66105dc9c", "tag": "3c705692-0c55-4ed6-88ed-b7ba46d3442f"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:24.919516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d7d1b269-3dee-468f-8ccc-19cc352487ad req-38f6c57f-baa1-42b3-8e30-fad58c860e11 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:24.919990 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d7d1b269-3dee-468f-8ccc-19cc352487ad req-38f6c57f-baa1-42b3-8e30-fad58c860e11 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:24.920413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d7d1b269-3dee-468f-8ccc-19cc352487ad req-38f6c57f-baa1-42b3-8e30-fad58c860e11 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:24.931573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-d7d1b269-3dee-468f-8ccc-19cc352487ad req-38f6c57f-baa1-42b3-8e30-fad58c860e11 service nova] Creating event network-changed:3c705692-0c55-4ed6-88ed-b7ba46d3442f for instance ac5527bc-4e21-4157-9415-41e66105dc9c on hv-cinder-84914 Oct 12 02:34:24.936375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-d7d1b269-3dee-468f-8ccc-19cc352487ad req-38f6c57f-baa1-42b3-8e30-fad58c860e11 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030137 Oct 12 02:34:24.937011 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 387/771] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:25.323621 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:25.324233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:25.330715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:25.331232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:25.331661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:25.332350 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:25.332827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:25.333250 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:25.566163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:25.576008 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c90de600-7e55-48c0-a263-b165bcf50739 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.257986 Oct 12 02:34:25.588725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 385/772] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:25 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:26.602027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:26.602712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:26.608737 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:26.609467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:26.609890 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:26.610562 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:26.611015 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:26.611425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:26.748824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:26.756851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-55f680fe-fcb8-4a01-8853-215a4c2198fb tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.159472 Oct 12 02:34:26.757385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 388/773] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:26 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:26.951024 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-c34bbb29-128c-400c-a838-b13b7995bb12 req-be35a3a8-817d-4ed7-86ab-926c62c7e924 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac5527bc-4e21-4157-9415-41e66105dc9c", "tag": "3c705692-0c55-4ed6-88ed-b7ba46d3442f"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:34:26.960778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c34bbb29-128c-400c-a838-b13b7995bb12 req-be35a3a8-817d-4ed7-86ab-926c62c7e924 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:26.961252 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c34bbb29-128c-400c-a838-b13b7995bb12 req-be35a3a8-817d-4ed7-86ab-926c62c7e924 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:26.961667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c34bbb29-128c-400c-a838-b13b7995bb12 req-be35a3a8-817d-4ed7-86ab-926c62c7e924 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:26.974079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-c34bbb29-128c-400c-a838-b13b7995bb12 req-be35a3a8-817d-4ed7-86ab-926c62c7e924 service nova] Creating event network-changed:3c705692-0c55-4ed6-88ed-b7ba46d3442f for instance ac5527bc-4e21-4157-9415-41e66105dc9c on hv-cinder-84914 Oct 12 02:34:26.979915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-c34bbb29-128c-400c-a838-b13b7995bb12 req-be35a3a8-817d-4ed7-86ab-926c62c7e924 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032929 Oct 12 02:34:26.980481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 386/774] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:34:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:34:27.780250 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:27.780886 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:27.787317 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:27.787801 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:27.788229 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:27.789012 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:27.789465 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:27.789884 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:27.933261 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:27.941328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b0763f7e-4056-431a-9fbe-df44cbbbdf33 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.165393 Oct 12 02:34:27.941879 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 389/775] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:27 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:28.965933 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:28.966645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:28.973328 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:28.973800 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:28.974224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:28.974898 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:28.975343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:28.975758 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:29.122136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:29.132400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d661ce42-63c6-4ad0-b3de-4d1028f9d0c0 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.171232 Oct 12 02:34:29.133183 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 387/776] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:28 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:30.157307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:30.157915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:30.164382 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:30.164902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:30.165336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:30.166020 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:30.166471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:30.166891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:30.309732 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:30.317951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-db069498-cd02-4e33-9239-f0ac547731fc tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.164563 Oct 12 02:34:30.318510 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 390/777] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:30 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:31.341095 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:31.341733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:31.348202 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:31.348688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:31.349299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:31.349990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:31.350447 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:31.350872 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:31.497017 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:31.505219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-736bbaab-fe61-4838-99dd-e484a70d8f1f tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.167980 Oct 12 02:34:31.505785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 388/778] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:31 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:32.529346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:32.529956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:32.536488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:32.537005 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:32.537439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:32.538135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:32.538587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:32.539008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:32.688181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:32.696320 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-99a7ed32-3580-4658-9cf4-984eed26e689 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.171555 Oct 12 02:34:32.696907 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 391/779] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:32 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:33.721557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:33.722164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:33.729461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:33.729943 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:33.730368 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:33.731064 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:33.731491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:33.731906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:33.734778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:33.876214 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:33.884318 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d004f833-de15-468f-9508-194124d5e31a tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.166597 Oct 12 02:34:33.885462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 389/780] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:33 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:34.907442 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:34.907963 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:34.914422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:34.914899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:34.915330 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:34.916519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:34.916995 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:34.917415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:35.068007 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:35.076960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-331f8973-33bc-42c6-a6f9-e39dc94f4a17 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.173409 Oct 12 02:34:35.077512 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 392/781] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:34 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:36.100605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:36.101236 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:36.108209 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:36.108688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:36.109462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:36.110144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:36.110587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:36.111001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:36.249963 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:36.258279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7af8fa8e-d8c0-4e33-969d-a6ec4d04e7ca tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.161696 Oct 12 02:34:36.258853 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 390/782] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:36 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:37.281640 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:37.282241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:37.288623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:37.289136 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:37.289569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:37.290281 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:37.290735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:37.291173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:37.437959 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:37.446109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-80cc12cd-f80d-49d1-a3a6-c1ee2eaad1f1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.168333 Oct 12 02:34:37.446660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 393/783] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:37 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:38.472131 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:38.472761 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:38.479239 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:38.479725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:38.480154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:38.480881 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:38.481335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:38.481757 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:38.625565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:38.633905 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c2f15563-6686-4d74-9996-a91e89eb35bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.165722 Oct 12 02:34:38.634969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 391/784] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:38 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:39.657785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:39.658389 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:39.664928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:39.665414 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:39.666417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:39.667107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:39.667557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:39.667976 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:39.813362 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:39.821481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-00cdaadf-c2fe-4fd6-9948-6aa35077e2bf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.167601 Oct 12 02:34:39.822032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 394/785] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:39 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:40.844777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:40.845423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:40.847806 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=66 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:40.852998 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:40.853531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:40.853960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:40.854663 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:40.855092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:40.855506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:40.994986 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:41.002938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0b6a6704-0f96-43a9-b511-6c174307e794 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1754 microversion: 2.1 time: 0.161980 Oct 12 02:34:41.003498 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 392/786] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:40 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1754 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:42.026896 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:42.027528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:42.034143 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:42.034625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:42.035059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:42.035744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:42.036196 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:42.036616 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:42.176241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:34:42.184296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6a44bf20-ced2-40df-93e6-d863752f7baf tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1757 microversion: 2.1 time: 0.161777 Oct 12 02:34:42.184868 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 395/787] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:34:42 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1757 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:34:58.872584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:58.873279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=23 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:34:58.971107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:34:58.971596 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:34:58.978004 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:58.978474 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:58.978900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:58.979573 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:34:58.980014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:34:58.980424 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:34:59.002928 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d942479a-79de-41f3-9423-605870213de5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/ips" status: 200 len: 149 microversion: 2.1 time: 0.136196 Oct 12 02:34:59.003460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 393/788] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:34:58 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/ips => generated 149 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:00.341543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6e4d5a59-b54a-4185-8bab-46cbe424895d"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:00.343830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:00.346121 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=23 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:00.346804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=88 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:00.353062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:00.353547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:00.353986 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:00.354673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:00.355129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:00.355551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:00.622966 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-407110ab-5905-41b7-b091-4bd3723a94b5 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.285686 Oct 12 02:35:00.623620 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 396/789] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:35:00 2022] POST /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments => generated 194 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:03.093118 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:03.093623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:03.100005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:03.100521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:03.100988 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:03.101676 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:03.102672 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:03.103093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:03.181429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lazy-loading 'services' on Instance uuid ac5527bc-4e21-4157-9415-41e66105dc9c {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:03.254402 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-10c5a338-b16d-402d-9361-e9f39b4f4b7b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "DELETE /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments/459cc4e1-2d7e-451d-b4d6-c7d0753355fd" status: 202 len: 0 microversion: 2.1 time: 0.165057 Oct 12 02:35:03.255048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 394/790] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:35:03 2022] DELETE /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments/459cc4e1-2d7e-451d-b4d6-c7d0753355fd => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:35:03.265400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:03.267746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:03.274895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:03.275461 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:03.275955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:03.276829 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:03.277283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:03.277706 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:03.306630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d56c0d58-f265-4599-854d-085c4ac908af tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046436 Oct 12 02:35:03.307191 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 397/791] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:03 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:04.320169 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:04.322536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:04.328690 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:04.329197 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:04.329628 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:04.330305 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:04.330754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:04.331198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:04.360845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-71d86fed-10ae-4556-967b-44528b4a91dd tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045375 Oct 12 02:35:04.361391 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 395/792] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:04 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:05.373847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:05.375908 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:05.382259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:05.382757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:05.383205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:05.383921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:05.384346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:05.384792 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:05.412662 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0928a698-6f0a-4c49-a3cb-2d646404c83d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042859 Oct 12 02:35:05.413224 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 398/793] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:05 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:05.499672 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:05.500190 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:05.506067 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:05.506550 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:05.506982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:05.507665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:05.508112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:05.508532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:05.530783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1dc950e3-2d77-4046-8b1e-3526ae197ebc tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/ips" status: 200 len: 151 microversion: 2.20 time: 0.084088 Oct 12 02:35:05.531322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 396/794] 10.222.0.23 () {60 vars in 1289 bytes} [Wed Oct 12 02:35:05 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/ips => generated 151 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:35:06.429254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:06.431372 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:06.437619 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:06.438098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:06.438530 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:06.439211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:06.439663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:06.440088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:06.469481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f6348a89-4bd2-493d-9971-201de05fdd37 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044146 Oct 12 02:35:06.470038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 399/795] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:06 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:07.404882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:07.405458 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:07.411849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:07.412330 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:07.412776 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:07.413504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:07.413973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:07.414435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:07.440982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lazy-loading 'flavor' on Instance uuid a61f7314-595f-4a3e-9288-0d25df1b9607 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:07.483666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:07.485858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:07.493939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:07.494430 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:07.495082 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:07.496130 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:07.496810 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:07.497506 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:07.530253 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-099d386d-ba84-4f61-b45b-ba6b977820f4 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051856 Oct 12 02:35:07.530805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 400/796] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:07 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:07.626064 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-67c0f8d1-a6a5-4336-b37a-f29b7a54cded tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/action" status: 202 len: 0 microversion: 2.20 time: 0.225758 Oct 12 02:35:07.626646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 397/797] 10.222.0.23 () {62 vars in 1319 bytes} [Wed Oct 12 02:35:07 2022] POST /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/action => generated 0 bytes in 227 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:07.635487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:07.636097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:07.643141 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:07.643630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:07.644066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:07.644787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:07.645247 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:07.645673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:07.776612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:07.785282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:07.786712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:07.787858 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a9f510c6-e89b-4cba-ba63-613207aa7d95 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.156871 Oct 12 02:35:07.788525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 401/798] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:07 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:08.157994 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6e4d5a59-b54a-4185-8bab-46cbe424895d"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:08.160347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:08.167128 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.167611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.168042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.168759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.169203 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.169626 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.248070 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Invalid volume: volume 6e4d5a59-b54a-4185-8bab-46cbe424895d is already attached to instances: 928a91d5-600b-4707-9428-2e716f411885 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: POST /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments HTTP/1.1 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 74 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:35:08.249829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {"volumeAttachment": {"volumeId": "6e4d5a59-b54a-4185-8bab-46cbe424895d"}} Oct 12 02:35:08.251274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Invalid volume: volume 6e4d5a59-b54a-4185-8bab-46cbe424895d is already attached to instances: 928a91d5-600b-4707-9428-2e716f411885 Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'server_id': '928a91d5-600b-4707-9428-2e716f411885', 'body': {'volumeAttachment': {'volumeId': '6e4d5a59-b54a-4185-8bab-46cbe424895d'}}} Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 6e4d5a59-b54a-4185-8bab-46cbe424895d is already attached to instances: 928a91d5-600b-4707-9428-2e716f411885 Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:08.252012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 6e4d5a59-b54a-4185-8bab-46cbe424895d is already attached to instances: 928a91d5-600b-4707-9428-2e716f411885 Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:08.253273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 6e4d5a59-b54a-4185-8bab-46cbe424895d is already attached to instances: 928a91d5-600b-4707-9428-2e716f411885 Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 400 to user: Invalid volume: volume 6e4d5a59-b54a-4185-8bab-46cbe424895d is already attached to instances: 928a91d5-600b-4707-9428-2e716f411885 {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-80866cd0-6a2b-4d22-8fac-9bc8f1327e9d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.097572 Oct 12 02:35:08.254506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 398/799] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:35:08 2022] POST /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments => generated 174 bytes in 99 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:35:08.315348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:08.315834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:08.321629 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.322610 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.323033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.323705 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.324150 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.324563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.403916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'services' on Instance uuid 928a91d5-600b-4707-9428-2e716f411885 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:08.475683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-782346a9-1409-4d36-b138-8bf34a17cf32 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments/6e4d5a59-b54a-4185-8bab-46cbe424895d" status: 202 len: 0 microversion: 2.1 time: 0.164021 Oct 12 02:35:08.476331 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 402/800] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:35:08 2022] DELETE /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments/6e4d5a59-b54a-4185-8bab-46cbe424895d => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:35:08.486000 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:08.488101 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:08.497019 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.497510 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.498515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.499719 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.500174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.500597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.531298 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1f79e6a1-d774-4855-8840-d1b20c47205f tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050536 Oct 12 02:35:08.531846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 399/801] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:08 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:08.543551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:08.545843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:08.552001 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.552485 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.552957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.553647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.554102 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.554527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.583499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-39d8fd3b-6ef6-40d9-90e2-5772b9796047 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044766 Oct 12 02:35:08.584059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 403/802] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:08 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:35:08.669948 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:08.670440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:08.676248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.676749 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.677182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.677858 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.678310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.678724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.703848 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:35:08.774514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-becc687d-3e0f-4284-bc6b-595d12360635 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "DELETE /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 204 len: 0 microversion: 2.1 time: 0.108286 Oct 12 02:35:08.775117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 400/803] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:35:08 2022] DELETE /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:08.782997 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:08.783621 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:08.789647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.790128 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.790556 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.791244 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.791694 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.792141 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.819011 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:08.819613 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:08.826003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.826486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.826919 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.827604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:08.828053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:08.828471 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:08.926408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:08.934876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-963f100c-d420-4a52-ac9a-f045416b8f54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1691 microversion: 2.1 time: 0.155666 Oct 12 02:35:08.935467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 404/804] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:08 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1691 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:08.961379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:08.969692 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:08.971247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:08.972504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-5d35f0ef-acef-4604-a41c-8bf64c3415e8 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.157743 Oct 12 02:35:08.973110 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 401/805] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:08 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:09.543611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:09.545895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:09.552435 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:09.552946 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:09.553377 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:09.554055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:09.554529 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:09.554952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:09.584188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-31e829b4-3ab0-4b9d-9aa8-07e7a44d7b90 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044569 Oct 12 02:35:09.584738 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 405/806] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:09 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:09.958540 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:09.959146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:09.965532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:09.966016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:09.966449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:09.967129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:09.967580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:09.967997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:09.998798 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:09.999406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:10.005348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:10.005839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:10.006275 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:10.006970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:10.007426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:10.007854 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:10.099130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:10.107372 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a03fd799-c936-458a-8b3e-5d514b092f35 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1691 microversion: 2.1 time: 0.152616 Oct 12 02:35:10.107934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 402/807] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:09 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1691 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:10.135978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:10.143934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:10.145409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:10.146541 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-46347d06-f998-4340-805f-f499b57b8306 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.151591 Oct 12 02:35:10.147083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 406/808] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:09 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:10.596836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:10.599103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:10.605004 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:10.605481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:10.605910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:10.606590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:10.607034 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:10.607453 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:10.635962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6510ab8e-cab0-4412-ae9c-617f9e84adf1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043120 Oct 12 02:35:10.636500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 403/809] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:10 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:11.131528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:11.132158 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:11.139471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:11.139949 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:11.140380 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:11.141087 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:11.141537 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:11.141950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:11.172813 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:11.173533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:11.180012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:11.180497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:11.180974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:11.182171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:11.182628 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:11.183054 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:11.274840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:11.282616 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6418ceff-cb47-473c-b059-632952863f9d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1691 microversion: 2.1 time: 0.155059 Oct 12 02:35:11.283182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 407/810] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:11 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1691 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:11.311795 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:11.319708 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:11.321151 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:11.322822 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-aeb35d45-eaba-477e-a7a2-0d5cc07c5067 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.153308 Oct 12 02:35:11.323369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 404/811] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:11 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:11.648276 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:11.651019 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:11.656855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:11.657333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:11.657756 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:11.658432 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:11.658876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:11.659290 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:11.688140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-615e1821-7bfd-4e47-be87-c021ef8f921d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043730 Oct 12 02:35:11.688738 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 408/812] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:11 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:12.305464 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:12.306065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:12.313557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.314035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.314465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.315146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.315590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.316374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.349066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:12.349698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:12.356618 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.357140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.357581 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.358546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.358999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.359420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.447848 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:12.451459 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:12.456962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f8173414-fc01-431b-9ccd-8abcfef2dd54 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1691 microversion: 2.1 time: 0.155210 Oct 12 02:35:12.457532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 405/813] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:12 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1691 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:12.491940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:12.500179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:12.501674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:12.502824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bd5d0a31-1ef6-4c71-920f-bc2a35853a4a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.157921 Oct 12 02:35:12.503383 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 409/814] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:12 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:12.701112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:12.703689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:12.711985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.712507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.712960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.714391 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.714847 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.715269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.750683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-799ff350-f524-4f2d-b64d-db32f0783e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053332 Oct 12 02:35:12.751601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 406/815] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:12 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:35:12.816499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:12.816983 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:12.823377 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.823879 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.824311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.825033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.825487 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.825908 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.844670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:12.852925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:35:12.930328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a55e8497-5f16-49f7-9f5a-20d2a0790e9a tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 204 len: 0 microversion: 2.1 time: 0.117296 Oct 12 02:35:12.931129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 410/816] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:35:12 2022] DELETE /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:12.941032 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:12.941640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:12.947655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.948666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.949121 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:12.949804 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:12.950255 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:12.950698 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:13.176001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:13.184938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1628eca3-5bc8-4f2e-bce2-6b78b5f5bcc8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1680 microversion: 2.1 time: 0.249340 Oct 12 02:35:13.185439 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 407/817] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:12 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1680 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:13.481782 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:13.482607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:13.489190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:13.489673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:13.490290 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:13.491164 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:13.491779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:13.492676 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:13.527874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:13.529126 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:13.538699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:13.539386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:13.540114 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:13.541508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:13.542816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:13.543396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:13.566246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:13.574460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-11c393a9-5af8-43fa-998e-d85bb9fd76df tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1623 microversion: 2.1 time: 0.096873 Oct 12 02:35:13.584827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 411/818] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:13 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1623 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:13.657051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-c3bd6592-776b-42e9-a85d-6fbdbbede2c8 req-3de8cf8f-f016-4e81-9668-1a4893b688a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac5527bc-4e21-4157-9415-41e66105dc9c", "name": "network-vif-deleted", "tag": "3c705692-0c55-4ed6-88ed-b7ba46d3442f"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:13.669137 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-c3bd6592-776b-42e9-a85d-6fbdbbede2c8 req-3de8cf8f-f016-4e81-9668-1a4893b688a1 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:13.669909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-c3bd6592-776b-42e9-a85d-6fbdbbede2c8 req-3de8cf8f-f016-4e81-9668-1a4893b688a1 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:13.670519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-c3bd6592-776b-42e9-a85d-6fbdbbede2c8 req-3de8cf8f-f016-4e81-9668-1a4893b688a1 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:13.683323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-c3bd6592-776b-42e9-a85d-6fbdbbede2c8 req-3de8cf8f-f016-4e81-9668-1a4893b688a1 service nova] Creating event network-vif-deleted:3c705692-0c55-4ed6-88ed-b7ba46d3442f for instance ac5527bc-4e21-4157-9415-41e66105dc9c on hv-cinder-84914 Oct 12 02:35:13.688743 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-c3bd6592-776b-42e9-a85d-6fbdbbede2c8 req-3de8cf8f-f016-4e81-9668-1a4893b688a1 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.112166 Oct 12 02:35:13.693418 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 412/819] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:13.703441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:13.712021 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:13.713544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:13.714711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-751d9e62-d824-441c-be86-540e7669d2e9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.191084 Oct 12 02:35:13.715268 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 408/820] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:13 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:14.210005 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:14.210693 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:14.218053 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:14.218622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:14.219143 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:14.219911 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:14.220436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:14.220969 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:14.354822 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:14.365746 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-098809b3-f368-44f2-a35d-f58f24fe4ab4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1680 microversion: 2.1 time: 0.159859 Oct 12 02:35:14.366444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 413/821] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:14 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:14.598507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:14.599128 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:14.605221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:14.605710 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:14.606149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:14.606839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:14.607296 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:14.607720 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:14.668462 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:14.676078 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4558fb3d-d0f8-4a7b-95b2-2edf27b920e7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 200 len: 1330 microversion: 2.1 time: 0.081423 Oct 12 02:35:14.676647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 409/822] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:14 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 1330 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:14.741488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:14.742132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:14.749067 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:14.749553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:14.749996 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:14.750683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:14.751139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:14.751562 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:14.877822 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:14.887060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:14.888498 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:14.889690 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-646f0208-8530-44db-ace7-e9a2a416a777 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.152079 Oct 12 02:35:14.890280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 414/823] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:14 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:15.390222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:15.390836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:15.396887 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:15.397389 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:15.397803 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:15.398502 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:15.398950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:15.399378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:15.700106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:15.700994 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: ac5527bc-4e21-4157-9415-41e66105dc9c] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:15.707080 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:15.707570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:15.708006 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:15.708725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:15.709195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:15.709648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:15.723218 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] HTTP exception thrown: Instance ac5527bc-4e21-4157-9415-41e66105dc9c could not be found. Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Request method failure captured: Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c HTTP/1.1 Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaPYwNdGPkaZC6cHEUGZ0SkhncrUI5nVnVZxOFbWJ3oJ3B77igC0roWc-nMKs5IUYGUtOa_YaPxwFn9fOjAYUswTJuudkNK6KicjYPQGN35ZLr1BK_R3Kt1jRPqGECcut8i4CjPKiklVRegxUQM3z5ZDQ5sJ8YSuV1B4cub9phJoCc Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:35:15.725028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bd41b04d606949edbc9a3a2f88f991b2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: bcd7f1121dab4e8db4c747c51ed49240 Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance ac5527bc-4e21-4157-9415-41e66105dc9c could not be found. Oct 12 02:35:15.726599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'ac5527bc-4e21-4157-9415-41e66105dc9c'} Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac5527bc-4e21-4157-9415-41e66105dc9c could not be found. Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:35:15.727560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac5527bc-4e21-4157-9415-41e66105dc9c could not be found. Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac5527bc-4e21-4157-9415-41e66105dc9c could not be found. Oct 12 02:35:15.729030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:15.731311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Returning 404 to user: Instance ac5527bc-4e21-4157-9415-41e66105dc9c could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:35:15.731311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9a8f8113-a2c8-4144-9082-f8859534020d tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c" status: 404 len: 111 microversion: 2.1 time: 0.030854 Oct 12 02:35:15.731311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 415/824] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:15 2022] GET /compute/v2.1/servers/ac5527bc-4e21-4157-9415-41e66105dc9c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:35:15.735323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bbcff81d-8e5c-4787-ab82-43bf97bb26c5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '.version_select at 0x7f59d345c160>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:15.757886 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bbcff81d-8e5c-4787-ab82-43bf97bb26c5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1012864248" status: 202 len: 0 microversion: 2.1 time: 0.026312 Oct 12 02:35:15.762526 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 416/825] 10.222.0.23 () {58 vars in 1221 bytes} [Wed Oct 12 02:35:15 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1012864248 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 12 02:35:15.822248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:15.838010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-67c5aa69-c4a3-455f-a2c7-22764ed5b8da tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 200 len: 1680 microversion: 2.1 time: 0.451637 Oct 12 02:35:15.849043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 410/826] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:15 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 1680 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:15.915431 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:15.916255 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:15.923841 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:15.924349 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:15.924801 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:15.925688 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:15.926153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:15.926558 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:16.062081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-139af47b-8f10-48f5-91c3-c0bdbaf0c9c5 req-d13b8eb8-c7a3-4c12-9a5f-c6a129b6f7d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "928a91d5-600b-4707-9428-2e716f411885", "name": "network-vif-deleted", "tag": "8ff097ef-11a1-4847-9e06-95f2c84807d0"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:16.075327 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-139af47b-8f10-48f5-91c3-c0bdbaf0c9c5 req-d13b8eb8-c7a3-4c12-9a5f-c6a129b6f7d2 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:16.075871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-139af47b-8f10-48f5-91c3-c0bdbaf0c9c5 req-d13b8eb8-c7a3-4c12-9a5f-c6a129b6f7d2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:16.076521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-139af47b-8f10-48f5-91c3-c0bdbaf0c9c5 req-d13b8eb8-c7a3-4c12-9a5f-c6a129b6f7d2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:16.094652 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-139af47b-8f10-48f5-91c3-c0bdbaf0c9c5 req-d13b8eb8-c7a3-4c12-9a5f-c6a129b6f7d2 service nova] Creating event network-vif-deleted:8ff097ef-11a1-4847-9e06-95f2c84807d0 for instance 928a91d5-600b-4707-9428-2e716f411885 on hv-cinder-84914 Oct 12 02:35:16.105850 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-139af47b-8f10-48f5-91c3-c0bdbaf0c9c5 req-d13b8eb8-c7a3-4c12-9a5f-c6a129b6f7d2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047228 Oct 12 02:35:16.106667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 411/827] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:16.282558 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:16.296526 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:16.299687 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:16.304600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-eb44518c-c47d-4a4a-9e9d-e26d8cf76a10 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.392979 Oct 12 02:35:16.306463 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 417/828] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:15 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:16.864712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:16.872350 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:16.879642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:16.880174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:16.880597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:16.881358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:16.881882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:16.882415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:16.896852 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 HTTP/1.1 Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:35:16.898679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:35:16.900071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '928a91d5-600b-4707-9428-2e716f411885'} Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:35:16.901454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:35:16.902680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:16.903716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:35:16.903716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8707d3a7-fc10-42b6-aa2c-a5ddb8e900ac tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 404 len: 111 microversion: 2.1 time: 0.040587 Oct 12 02:35:16.903716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 412/829] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:16 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:35:16.909873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-65250e7c-12d1-41a4-b9dd-ac9a22474815 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '.version_select at 0x7f59d345c4c0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:16.931138 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-65250e7c-12d1-41a4-b9dd-ac9a22474815 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-270069979" status: 202 len: 0 microversion: 2.1 time: 0.025217 Oct 12 02:35:16.932268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 418/830] 10.222.0.23 () {58 vars in 1218 bytes} [Wed Oct 12 02:35:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-270069979 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:35:17.333233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:17.333828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:17.339930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:17.340411 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:17.340871 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:17.341550 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:17.341997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:17.342412 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:17.532080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:17.540165 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:17.542189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:17.543384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d7cf0b58-ae10-4f7b-96ff-c7eefa646215 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.213971 Oct 12 02:35:17.543873 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 413/831] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:17 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:17.634370 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-05244497-dad1-45a2-b614-3375fbbf7ed7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: .version_select at 0x7f59d3515d30>, body: {"keypair": {"name": "tempest-keypair-1738962876", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDx1x65Qsj2rtcsjDM71PUw+Ab6ysysxJA9zZ0kLNT5ayk3q79/n18vDu8TtxIGF2aZO2fQRsOemZo+GIDEc/IDS3UXwGIFWpAlRzO2UHeO2/VDW2LoraLoVRKWOVDfLCA=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:17.653244 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-05244497-dad1-45a2-b614-3375fbbf7ed7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:35:17.657960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-05244497-dad1-45a2-b614-3375fbbf7ed7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:35:17.681475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-05244497-dad1-45a2-b614-3375fbbf7ed7 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.050983 Oct 12 02:35:17.682319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 419/832] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:35:17 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:18.574108 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:18.574772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:18.581565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:18.582089 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:18.582556 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:18.583254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:18.583704 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:18.584119 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:18.715632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:18.723690 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:18.725161 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:18.726319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2b1881b6-ce12-4340-b7f7-0aa6c4d30afb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.156978 Oct 12 02:35:18.726881 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 414/833] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:18 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:18.835472 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:18.836224 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:18.843110 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:18.843698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:18.844203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:18.845021 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:18.845548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:18.846041 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:18.873891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:35:18.958455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-152fa7ad-0ea5-426e-b4dd-95053516e1b6 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "DELETE /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 204 len: 0 microversion: 2.1 time: 0.126150 Oct 12 02:35:18.963030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 420/834] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:35:18 2022] DELETE /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:18.968365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:18.969007 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:18.975291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:18.975783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:18.976206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:18.976933 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:18.977491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:18.977908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:19.112246 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:19.129093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-25ea5456-a387-420a-bedb-51194f7ae87b tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1508 microversion: 2.1 time: 0.164588 Oct 12 02:35:19.129684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 415/835] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:18 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1508 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:35:19.174957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-416051643", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "21fdf3f0-6b90-400a-b2fc-5e540c8c18e8"}], "security_groups": [{"name": "tempest-securitygroup--1569390675"}], "key_name": "tempest-keypair-1738962876", "user_data": "}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:19.277032 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] validate_networks() for [('21fdf3f0-6b90-400a-b2fc-5e540c8c18e8', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:35:19.681511 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:35:19.681932 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:35:19.682427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:35:19.682870 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:35:19.683283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:35:19.691925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:19.692399 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:19.692880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:19.693600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:19.694051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:19.694471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:19.711112 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:35:19.715516 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Getting quotas for user 1fceec891ef24ba3a39da72c90fb733a and project 76e921b067f04677b76c245b5f93e112. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:35:19.726162 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:35:19.728478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:35:19.753131 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:19.753813 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:19.761677 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:19.762235 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:19.762734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:19.763479 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:19.764010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:19.764486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:19.775289 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-12c9dccf-5ab4-4e40-a0f0-2b59a6d254b9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.604878 Oct 12 02:35:19.776528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 421/836] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:35:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 607 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:35:19.787717 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-12eac5f0-5e41-4d13-8cc5-7f4debc7261e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:19.788332 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-12eac5f0-5e41-4d13-8cc5-7f4debc7261e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:19.845796 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-12eac5f0-5e41-4d13-8cc5-7f4debc7261e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:19.853059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-12eac5f0-5e41-4d13-8cc5-7f4debc7261e tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1241 microversion: 2.1 time: 0.069125 Oct 12 02:35:19.853942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 422/837] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:19 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1241 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:35:19.901938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:19.909970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:19.911405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:19.912544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1743f05e-e53f-4b5c-9f50-4ff95a723928 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.164061 Oct 12 02:35:19.913133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 416/838] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:19 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:20.151600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:20.152191 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:20.155321 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:20.159492 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:20.159979 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:20.160405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:20.161113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:20.161561 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:20.161976 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:20.305066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:20.313232 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d6ccb739-f17b-40cd-b168-bdf7361f1a88 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1508 microversion: 2.1 time: 0.165778 Oct 12 02:35:20.313789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 423/839] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:20 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1508 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:20.875683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:20.876497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:20.883166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:20.883730 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:20.884894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:20.885668 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:20.886350 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:20.887537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:20.938934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:20.939528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:20.945966 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:20.946533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:20.946960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:20.947638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:20.948085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:20.948500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:20.958158 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:20.967518 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-42e96601-e9db-4e5a-995c-039f50b00eb1 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1293 microversion: 2.1 time: 0.095851 Oct 12 02:35:20.968216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 417/840] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:20 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1293 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:21.079428 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:21.087360 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:21.089494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:21.090646 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d513aca1-87fb-4b57-a14b-5a379a047a21 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.156409 Oct 12 02:35:21.091190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 424/841] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:20 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:21.216276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-b4dee4d3-9569-4fc9-be50-09e2339876b3 req-79c70567-5397-42af-8e1a-f2c7437e7d53 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "44170331-47f8-456c-ad56-e86eab5cac15", "name": "network-vif-deleted", "tag": "083d5d6b-4001-402b-be73-2549ed579b58"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:21.227547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b4dee4d3-9569-4fc9-be50-09e2339876b3 req-79c70567-5397-42af-8e1a-f2c7437e7d53 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:21.228015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b4dee4d3-9569-4fc9-be50-09e2339876b3 req-79c70567-5397-42af-8e1a-f2c7437e7d53 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:21.228641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b4dee4d3-9569-4fc9-be50-09e2339876b3 req-79c70567-5397-42af-8e1a-f2c7437e7d53 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:21.243487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-b4dee4d3-9569-4fc9-be50-09e2339876b3 req-79c70567-5397-42af-8e1a-f2c7437e7d53 service nova] Creating event network-vif-deleted:083d5d6b-4001-402b-be73-2549ed579b58 for instance 44170331-47f8-456c-ad56-e86eab5cac15 on hv-cinder-84914 Oct 12 02:35:21.249392 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-b4dee4d3-9569-4fc9-be50-09e2339876b3 req-79c70567-5397-42af-8e1a-f2c7437e7d53 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037136 Oct 12 02:35:21.250680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 418/842] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:21.335597 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:21.336261 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:21.343105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:21.343905 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:21.344568 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:21.345610 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:21.346338 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:21.346865 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:21.384415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:21.385485 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:21.386050 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:21.386502 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lazy-loading 'fault' on Instance uuid 44170331-47f8-456c-ad56-e86eab5cac15 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:21.427647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:21.436241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ab89247c-791f-4858-a566-525cca77fa43 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 200 len: 1312 microversion: 2.1 time: 0.104655 Oct 12 02:35:21.436833 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 425/843] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:21 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 1312 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:21.995340 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:21.996580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:22.004549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.005154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.005595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.006302 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.006843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.007508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.116910 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:22.117570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:22.127829 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.128462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.129112 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.129813 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.130263 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.130681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.220488 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:22.229783 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f708f497-9238-4a19-bc01-84606c1c9075 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.240860 Oct 12 02:35:22.230799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 419/844] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:21 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:22.292249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:22.300320 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:22.301793 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:22.302949 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-20617197-92e4-43ee-959f-16315a621161 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.190496 Oct 12 02:35:22.303501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 426/845] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:22 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:22.459375 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:22.460133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 44170331-47f8-456c-ad56-e86eab5cac15] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:22.466259 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.466744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.467240 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.467863 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.468317 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.468759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.483622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] HTTP exception thrown: Instance 44170331-47f8-456c-ad56-e86eab5cac15 could not be found. Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Request method failure captured: Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 HTTP/1.1 Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaPYwNdGPkaZC6cHEUGZ0SkhncrUI5nVnVZxOFbWJ3oJ3B77igC0roWc-nMKs5IUYGUtOa_YaPxwFn9fOjAYUswTJuudkNK6KicjYPQGN35ZLr1BK_R3Kt1jRPqGECcut8i4CjPKiklVRegxUQM3z5ZDQ5sJ8YSuV1B4cub9phJoCc Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:35:22.485306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bd41b04d606949edbc9a3a2f88f991b2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: bcd7f1121dab4e8db4c747c51ed49240 Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 44170331-47f8-456c-ad56-e86eab5cac15 could not be found. Oct 12 02:35:22.486571 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '44170331-47f8-456c-ad56-e86eab5cac15'} Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 44170331-47f8-456c-ad56-e86eab5cac15 could not be found. Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:35:22.487434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 44170331-47f8-456c-ad56-e86eab5cac15 could not be found. Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 44170331-47f8-456c-ad56-e86eab5cac15 could not be found. Oct 12 02:35:22.488564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:22.490953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Returning 404 to user: Instance 44170331-47f8-456c-ad56-e86eab5cac15 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:35:22.490953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e489421c-81eb-403e-8d3c-07d88b36d839 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15" status: 404 len: 111 microversion: 2.1 time: 0.031175 Oct 12 02:35:22.490953 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 420/846] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:22 2022] GET /compute/v2.1/servers/44170331-47f8-456c-ad56-e86eab5cac15 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:35:22.496236 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:22.496757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:22.502821 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.504625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.504625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.504625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.504971 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.505397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.533511 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:35:22.586805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:22.615429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5974b4de-aa69-4f49-a6ef-70990d1cbba5 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 204 len: 0 microversion: 2.1 time: 0.123661 Oct 12 02:35:22.616063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 427/847] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:35:22 2022] DELETE /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:22.624990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:22.625595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:22.631340 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.631815 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.632241 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.633035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:22.633493 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:22.633910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:22.839988 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:22.847954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-513b8a6e-f778-4a18-aa2e-72c42985daed tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.226891 Oct 12 02:35:22.848522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 421/848] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:22 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:35:23.253668 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:23.254386 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:23.261325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.261809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.262240 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.262925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.263380 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.263802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.272441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-efabf0b9-babf-4341-b2b3-1f2070751e4e req-6513ee0c-04af-4716-afb5-c4471f503942 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b142a74-f235-4ea2-b553-6a0e8bac1c91", "tag": "93c61a39-b587-4e1c-9122-a17862f7c444"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:23.282021 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-efabf0b9-babf-4341-b2b3-1f2070751e4e req-6513ee0c-04af-4716-afb5-c4471f503942 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.282492 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-efabf0b9-babf-4341-b2b3-1f2070751e4e req-6513ee0c-04af-4716-afb5-c4471f503942 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.282918 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-efabf0b9-babf-4341-b2b3-1f2070751e4e req-6513ee0c-04af-4716-afb5-c4471f503942 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.293838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-efabf0b9-babf-4341-b2b3-1f2070751e4e req-6513ee0c-04af-4716-afb5-c4471f503942 service nova] Creating event network-changed:93c61a39-b587-4e1c-9122-a17862f7c444 for instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 on hv-cinder-84914 Oct 12 02:35:23.301886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-efabf0b9-babf-4341-b2b3-1f2070751e4e req-6513ee0c-04af-4716-afb5-c4471f503942 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032971 Oct 12 02:35:23.302482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 422/849] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:23.328942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:23.329883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:23.336719 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.337225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.337655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.338395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.338846 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.339267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.395999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:23.403654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4251004f-5806-4c5b-ba44-0f0847699a10 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.154321 Oct 12 02:35:23.404462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 428/850] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:23 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:23.470954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:23.479387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:23.482003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:23.483636 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-30a6980d-9a88-4e2a-992e-796c818186e2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.158809 Oct 12 02:35:23.484310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 423/851] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:23 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:23.621892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-514024739"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:23.622423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:23.628402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.628927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.629365 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.630055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.630510 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.630931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.757338 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.network.security_group_api [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Removing security group 6a67273a-7bab-41b0-a78f-cb2f6a061cad from port 299443a4-87d4-4257-8710-d074d71d9851 Oct 12 02:35:23.871603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:23.872316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:23.878253 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.878803 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.879238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:23.879910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:23.880352 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:23.880795 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:24.012417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:24.020096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0fbd89d6-d382-46a8-a7b3-0bddf88b63e3 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.152512 Oct 12 02:35:24.020688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 424/852] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:23 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:35:24.066417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e68ee476-c379-4448-b0e6-dda126030c26 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/action" status: 202 len: 0 microversion: 2.1 time: 0.449009 Oct 12 02:35:24.067030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 429/853] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:35:23 2022] POST /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/action => generated 0 bytes in 450 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:35:24.159343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-ee951fea-5d4b-499c-9d85-c4d9447c8d1f req-320a96b7-0ef1-44c7-8274-7f708dec6931 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bacd4ab7-51ad-4943-9d16-92bd1fc1adfa", "tag": "299443a4-87d4-4257-8710-d074d71d9851"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:24.169014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-ee951fea-5d4b-499c-9d85-c4d9447c8d1f req-320a96b7-0ef1-44c7-8274-7f708dec6931 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:24.169496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-ee951fea-5d4b-499c-9d85-c4d9447c8d1f req-320a96b7-0ef1-44c7-8274-7f708dec6931 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:24.169920 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-ee951fea-5d4b-499c-9d85-c4d9447c8d1f req-320a96b7-0ef1-44c7-8274-7f708dec6931 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:24.182523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-ee951fea-5d4b-499c-9d85-c4d9447c8d1f req-320a96b7-0ef1-44c7-8274-7f708dec6931 service nova] Creating event network-changed:299443a4-87d4-4257-8710-d074d71d9851 for instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa on hv-cinder-84914 Oct 12 02:35:24.187418 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-ee951fea-5d4b-499c-9d85-c4d9447c8d1f req-320a96b7-0ef1-44c7-8274-7f708dec6931 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.116420 Oct 12 02:35:24.188319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 425/854] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:24.428207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:24.430883 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:24.439982 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:24.440685 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:24.441302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:24.442088 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:24.442750 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:24.443402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:24.511186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:24.511974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:24.519714 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:24.520594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:24.521107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:24.521994 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:24.522679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:24.523162 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:24.544451 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:24.662060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:24.670304 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:24.671878 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:24.673066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1a387069-b179-415d-9553-e79b6d9fcd29 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.166713 Oct 12 02:35:24.673865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 426/855] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:24 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:24.675787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:24.683832 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ed26caea-f41f-4b2f-8b2e-0979f370d3c8 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.260491 Oct 12 02:35:24.684580 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 430/856] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:24 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:25.047640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:25.048719 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:25.055159 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.055733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.056324 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.057799 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.058333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.058831 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.388111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:25.398544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-379055d3-28d0-4b5f-9e5e-3996b22e1ab7 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1523 microversion: 2.1 time: 0.357809 Oct 12 02:35:25.399391 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 427/857] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:25 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1523 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:25.413333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:25.414082 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:25.421126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.421804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.422413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.423447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.424166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.424765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.521206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lazy-loading 'services' on Instance uuid bacd4ab7-51ad-4943-9d16-92bd1fc1adfa {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:25.612328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e118e57d-7997-48da-a7db-2f61658805a2 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "DELETE /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments/41cb5200-afd2-41c4-af92-d2fba5384e1a" status: 202 len: 0 microversion: 2.1 time: 0.202721 Oct 12 02:35:25.613762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 431/858] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:35:25 2022] DELETE /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments/41cb5200-afd2-41c4-af92-d2fba5384e1a => generated 0 bytes in 205 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:35:25.622929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:25.626096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:25.635720 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.636724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.637485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.638411 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.639077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.639775 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.679088 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1c1f3182-a008-4e41-8747-3eee96b3ebb1 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060856 Oct 12 02:35:25.680643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 428/859] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:25 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:25.709577 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:25.710798 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:25.722229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:25.722743 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.722872 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:25.723636 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.724297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.725461 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.726630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.727325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.729787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.730279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.730706 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.731401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:25.731850 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:25.732266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:25.948539 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:25.956641 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:25.958432 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:25.959595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-06f0e48e-74e3-450f-9d13-b8110010e09e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1742 microversion: 2.20 time: 0.255794 Oct 12 02:35:25.960457 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 432/860] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:25 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1742 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:25.992049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-dda84243-2ffb-41af-80f7-c596f5ca58c0 req-881a814a-42e1-4fbb-9aba-b2967f61ecae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d9e5223-f388-4f16-b15a-73c59e86de18", "name": "network-vif-deleted", "tag": "1ebc7cae-9780-4ccb-8ae6-cdd2a9b8caee"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:26.002171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-dda84243-2ffb-41af-80f7-c596f5ca58c0 req-881a814a-42e1-4fbb-9aba-b2967f61ecae service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.002768 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-dda84243-2ffb-41af-80f7-c596f5ca58c0 req-881a814a-42e1-4fbb-9aba-b2967f61ecae service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.003267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-dda84243-2ffb-41af-80f7-c596f5ca58c0 req-881a814a-42e1-4fbb-9aba-b2967f61ecae service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.015153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-dda84243-2ffb-41af-80f7-c596f5ca58c0 req-881a814a-42e1-4fbb-9aba-b2967f61ecae service nova] Creating event network-vif-deleted:1ebc7cae-9780-4ccb-8ae6-cdd2a9b8caee for instance 3d9e5223-f388-4f16-b15a-73c59e86de18 on hv-cinder-84914 Oct 12 02:35:26.020265 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-dda84243-2ffb-41af-80f7-c596f5ca58c0 req-881a814a-42e1-4fbb-9aba-b2967f61ecae service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031748 Oct 12 02:35:26.020918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 433/861] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:26.047938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:26.055773 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c8ec9b63-0ec7-4886-b8c6-ddb46522b6b6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.345843 Oct 12 02:35:26.056370 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 429/862] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:25 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:35:26.209701 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-bda88238-c3b1-4495-b4dc-9b6cc1c63414 req-138f067b-9bb9-410e-9a31-3d1bede92ad2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bacd4ab7-51ad-4943-9d16-92bd1fc1adfa", "tag": "299443a4-87d4-4257-8710-d074d71d9851"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:26.220366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-bda88238-c3b1-4495-b4dc-9b6cc1c63414 req-138f067b-9bb9-410e-9a31-3d1bede92ad2 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.221405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-bda88238-c3b1-4495-b4dc-9b6cc1c63414 req-138f067b-9bb9-410e-9a31-3d1bede92ad2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.222097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-bda88238-c3b1-4495-b4dc-9b6cc1c63414 req-138f067b-9bb9-410e-9a31-3d1bede92ad2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.236785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-bda88238-c3b1-4495-b4dc-9b6cc1c63414 req-138f067b-9bb9-410e-9a31-3d1bede92ad2 service nova] Creating event network-changed:299443a4-87d4-4257-8710-d074d71d9851 for instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa on hv-cinder-84914 Oct 12 02:35:26.244354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-bda88238-c3b1-4495-b4dc-9b6cc1c63414 req-138f067b-9bb9-410e-9a31-3d1bede92ad2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038826 Oct 12 02:35:26.245389 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 434/863] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:26.427048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:26.427663 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:26.435992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.436605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.436985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.437683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.438314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.438744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.475290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.476798 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.476798 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.476798 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lazy-loading 'fault' on Instance uuid 3d9e5223-f388-4f16-b15a-73c59e86de18 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:26.523817 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:26.532765 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9e893b27-cd14-4dfa-9a9d-3f3fb60359ae tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1312 microversion: 2.1 time: 0.109585 Oct 12 02:35:26.533340 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 430/864] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:26 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1312 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:26.693802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:26.696024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:26.702700 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.703172 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.703595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.704267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.704724 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.705146 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.732528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5ec58535-3636-4494-9ffd-d435f8957fcd tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043573 Oct 12 02:35:26.733350 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 435/865] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:26 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:26.985501 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:26.986100 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:26.992460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.992976 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.993407 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:26.994587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:26.995030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:26.995443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.079556 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:27.080460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:27.086757 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.087538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.087953 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.088903 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.089445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.089871 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.374469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:27.375286 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:27.382454 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3081a391-6262-41ac-95f7-e27be62baa61 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.306789 Oct 12 02:35:27.383024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 436/866] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:27 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:27.383867 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:27.385510 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:27.386691 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-82cb0fcd-c193-4cc2-b604-a1f460f13b3a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1742 microversion: 2.20 time: 0.405305 Oct 12 02:35:27.387231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 431/867] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:26 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1742 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 12 02:35:27.555221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:27.555825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:27.561698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.562204 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.562640 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.563324 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.563777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.564200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.594140 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.594615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.595045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.595480 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lazy-loading 'fault' on Instance uuid 3d9e5223-f388-4f16-b15a-73c59e86de18 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:27.635239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcd7f1121dab4e8db4c747c51ed49240', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd41b04d606949edbc9a3a2f88f991b2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:27.642850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1c4649ad-70e5-43ca-a720-b94a9bb539a9 tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 200 len: 1312 microversion: 2.1 time: 0.091467 Oct 12 02:35:27.643407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 437/868] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:27 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 1312 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:27.745832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:27.748439 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:27.754492 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.755232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.755665 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.756353 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.756982 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.757763 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.790722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e0f9479e-42af-44b5-89a1-177dafd9afc4 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048717 Oct 12 02:35:27.791614 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 432/869] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:35:27 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:35:27.805892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:27.808415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:27.821492 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.822849 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.823504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.825172 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:27.825837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:27.826270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:27.913839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] HTTP exception thrown: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa is not attached to volume 41cb5200-afd2-41c4-af92-d2fba5384e1a Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Request method failure captured: Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments/41cb5200-afd2-41c4-af92-d2fba5384e1a HTTP/1.1 Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRidSyUKq-WdBdCxWbgglikMhhfYz1_750aRJ3JNLnVpQI2EhxeySVDD56d5zYyPS1k5psNp39aBg5ZtgARA2ju8S3ab1i23JMhVJihs9X8VwifkKjjY_4Dd9VHyHWiqbCU71MOBNizxb-0A-z6bcsNz-kBV7GgNN0EbgHWavtE9JBiY Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: c57365923b844fe9b29d9e223ad49ce4 Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-TestMinimumBasicScenario-608340467 Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:35:27.916339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c57365923b844fe9b29d9e223ad49ce4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c57365923b844fe9b29d9e223ad49ce4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c57365923b844fe9b29d9e223ad49ce4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c57365923b844fe9b29d9e223ad49ce4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-TestMinimumBasicScenario-608340467 Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: c57365923b844fe9b29d9e223ad49ce4 Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-TestMinimumBasicScenario-608340467 Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-TestMinimumBasicScenario-608340467-project Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 945de18174894cd683ebf34372e845f0 Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-TestMinimumBasicScenario-608340467-project Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:35:27.917785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa is not attached to volume 41cb5200-afd2-41c4-af92-d2fba5384e1a Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'server_id': 'bacd4ab7-51ad-4943-9d16-92bd1fc1adfa', 'id': '41cb5200-afd2-41c4-af92-d2fba5384e1a'} Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 41cb5200-afd2-41c4-af92-d2fba5384e1a. Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:35:27.918904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa is not attached to volume 41cb5200-afd2-41c4-af92-d2fba5384e1a Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa is not attached to volume 41cb5200-afd2-41c4-af92-d2fba5384e1a Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:35:27.920113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Returning 404 to user: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa is not attached to volume 41cb5200-afd2-41c4-af92-d2fba5384e1a {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:35:27.921249 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-055ba023-1ff2-4e3f-be8c-f35e1dc71fad tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "DELETE /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments/41cb5200-afd2-41c4-af92-d2fba5384e1a" status: 404 len: 154 microversion: 2.1 time: 0.119703 Oct 12 02:35:27.922275 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 438/870] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:35:27 2022] DELETE /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa/os-volume_attachments/41cb5200-afd2-41c4-af92-d2fba5384e1a => generated 154 bytes in 125 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:35:28.404717 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:28.405337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:28.413883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:28.414377 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:28.415063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:28.415519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:28.417348 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:28.420500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:28.421176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:28.421860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:28.422352 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:28.423259 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:28.423674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:28.424438 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:28.425002 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:28.425436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:28.593167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:28.596670 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:28.605337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-345b1f86-454f-4355-9199-95ebda319ca3 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.202686 Oct 12 02:35:28.606212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 433/871] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:28 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:28.608351 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:28.610127 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:28.611615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b4ed7a38-56f6-42b3-858f-4d82ebda17b2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.203186 Oct 12 02:35:28.612812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 439/872] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:28 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:28.666740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:28.667646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] [instance: 3d9e5223-f388-4f16-b15a-73c59e86de18] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:28.673913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:28.674802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:28.675735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:28.676872 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:28.677878 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:28.678810 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:28.696872 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] HTTP exception thrown: Instance 3d9e5223-f388-4f16-b15a-73c59e86de18 could not be found. Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Request method failure captured: Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 HTTP/1.1 Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaPYwNdGPkaZC6cHEUGZ0SkhncrUI5nVnVZxOFbWJ3oJ3B77igC0roWc-nMKs5IUYGUtOa_YaPxwFn9fOjAYUswTJuudkNK6KicjYPQGN35ZLr1BK_R3Kt1jRPqGECcut8i4CjPKiklVRegxUQM3z5ZDQ5sJ8YSuV1B4cub9phJoCc Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:35:28.698647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bd41b04d606949edbc9a3a2f88f991b2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bd41b04d606949edbc9a3a2f88f991b2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: bd41b04d606949edbc9a3a2f88f991b2 Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-897468484 Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: bcd7f1121dab4e8db4c747c51ed49240 Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-ServerRescueNegativeTestJSON-897468484-project Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 3d9e5223-f388-4f16-b15a-73c59e86de18 could not be found. Oct 12 02:35:28.700176 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '3d9e5223-f388-4f16-b15a-73c59e86de18'} Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3d9e5223-f388-4f16-b15a-73c59e86de18 could not be found. Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:35:28.701090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3d9e5223-f388-4f16-b15a-73c59e86de18 could not be found. Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3d9e5223-f388-4f16-b15a-73c59e86de18 could not be found. Oct 12 02:35:28.702551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:28.704477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] Returning 404 to user: Instance 3d9e5223-f388-4f16-b15a-73c59e86de18 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:35:28.705343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f28e6ce0-1a98-4c50-9bf4-edef7383378c tempest-ServerRescueNegativeTestJSON-897468484 tempest-ServerRescueNegativeTestJSON-897468484-project] 10.222.0.23 "GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18" status: 404 len: 111 microversion: 2.1 time: 0.043277 Oct 12 02:35:28.709310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 434/873] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:28 2022] GET /compute/v2.1/servers/3d9e5223-f388-4f16-b15a-73c59e86de18 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:35:29.084035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:29.086113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:29.110117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.110956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.111566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.117540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.118216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.118800 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.158469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:35:29.301213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4d0b3dcf-faef-44b0-8400-37425134f711 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "DELETE /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 204 len: 0 microversion: 2.1 time: 0.223369 Oct 12 02:35:29.302324 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 440/874] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:35:29 2022] DELETE /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 0 bytes in 226 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:29.314195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:29.315204 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:29.322174 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.322940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.323660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.324735 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.325341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.325909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.350143 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-c5769083-3c44-4037-86c2-07d6dbb3cda7 req-0f09b2ea-a300-4f61-8537-90affc602713 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a61f7314-595f-4a3e-9288-0d25df1b9607", "name": "network-vif-plugged", "status": "completed", "tag": "fe5a040d-6b9b-4b90-a3c9-76a1085a756e"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:29.355908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:29.379866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c5769083-3c44-4037-86c2-07d6dbb3cda7 req-0f09b2ea-a300-4f61-8537-90affc602713 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.380391 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c5769083-3c44-4037-86c2-07d6dbb3cda7 req-0f09b2ea-a300-4f61-8537-90affc602713 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.380870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c5769083-3c44-4037-86c2-07d6dbb3cda7 req-0f09b2ea-a300-4f61-8537-90affc602713 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.426048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-c5769083-3c44-4037-86c2-07d6dbb3cda7 req-0f09b2ea-a300-4f61-8537-90affc602713 service nova] Creating event network-vif-plugged:fe5a040d-6b9b-4b90-a3c9-76a1085a756e for instance a61f7314-595f-4a3e-9288-0d25df1b9607 on hv-cinder-84914 Oct 12 02:35:29.433371 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-c5769083-3c44-4037-86c2-07d6dbb3cda7 req-0f09b2ea-a300-4f61-8537-90affc602713 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.201153 Oct 12 02:35:29.434794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 435/875] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:29.631319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:29.632644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:29.639558 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.640412 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.641130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.642079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.642997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.643666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.702697 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:29.850348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=12 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:29.858481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:29.867577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7f5bd28b-cbd2-41cf-8814-91a0c0672085 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.241622 Oct 12 02:35:29.872485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 436/876] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:29 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:29.880117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:29.881096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:29.895234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.899124 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.899594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.903031 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:29.903532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:29.903968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:29.925739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:29.935364 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-62e4e420-dc61-47b2-b0ce-3ab4054d728c tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1553 microversion: 2.1 time: 0.626630 Oct 12 02:35:29.935936 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 441/877] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:29 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1553 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:30.084661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:30.098168 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:30.099949 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:30.103395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-884504d3-b952-4a11-b219-d2a641acb4c4 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.227201 Oct 12 02:35:30.104133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 437/878] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:29 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:30.894661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:30.895893 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:30.902834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:30.903336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:30.903771 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:30.904478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:30.904957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:30.905537 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:30.959079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:30.959680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:30.969795 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:30.970566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:30.972192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:30.973475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:30.974518 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:30.975764 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:31.131883 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:31.145206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-348d3f03-e900-4f44-b29c-91cef2ba46af tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.257545 Oct 12 02:35:31.149683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 442/879] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:30 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:31.180970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:31.183135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:31.197275 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:31.198829 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:31.200928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:31.202739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:31.204623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:31.206806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:31.248919 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:31.267851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:31.276440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-98344a47-42db-4544-b740-8022933866d5 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1553 microversion: 2.1 time: 0.320232 Oct 12 02:35:31.281355 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 438/880] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:30 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1553 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:31.491555 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-321259b9-2ee3-437a-984c-1fc61d59921a req-c28ff4fb-2752-496c-8c0c-8b8de97825e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b142a74-f235-4ea2-b553-6a0e8bac1c91", "name": "network-vif-plugged", "status": "completed", "tag": "93c61a39-b587-4e1c-9122-a17862f7c444"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:31.520300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-321259b9-2ee3-437a-984c-1fc61d59921a req-c28ff4fb-2752-496c-8c0c-8b8de97825e4 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:31.521105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-321259b9-2ee3-437a-984c-1fc61d59921a req-c28ff4fb-2752-496c-8c0c-8b8de97825e4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:31.521246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:31.522225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-321259b9-2ee3-437a-984c-1fc61d59921a req-c28ff4fb-2752-496c-8c0c-8b8de97825e4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:31.530804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:31.532643 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:31.534107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-eed16ae7-1a29-41a0-bf0e-060f683bc996 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.361295 Oct 12 02:35:31.549891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 443/881] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:31 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:31.552219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-321259b9-2ee3-437a-984c-1fc61d59921a req-c28ff4fb-2752-496c-8c0c-8b8de97825e4 service nova] Creating event network-vif-plugged:93c61a39-b587-4e1c-9122-a17862f7c444 for instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 on hv-cinder-84914 Oct 12 02:35:31.576531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-321259b9-2ee3-437a-984c-1fc61d59921a req-c28ff4fb-2752-496c-8c0c-8b8de97825e4 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083520 Oct 12 02:35:31.577811 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 439/882] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:32.177536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:32.178794 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:32.193961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:32.194744 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:32.195332 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:32.196208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:32.196922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:32.197500 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:32.313488 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:32.315830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:32.324028 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:32.324992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:32.325671 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:32.326779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:32.327410 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:32.327997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:32.463249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:32.477738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2597141a-8440-4945-ad11-dfbadbdb60d9 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1511 microversion: 2.1 time: 0.170392 Oct 12 02:35:32.498886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 440/883] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:32 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1511 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:32.564579 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:32.565191 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:32.572577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:32.573186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:32.573839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:32.574903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:32.575554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:32.576170 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:32.644097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:32.673586 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-79eba7a4-a3e8-4ab5-b98b-b93015b70b94 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1367 microversion: 2.1 time: 0.491609 Oct 12 02:35:32.700060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 444/884] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:32 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1367 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:32.713648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-14846c21-9af3-4c3a-a569-e539eaa873a9 req-c9bd9cf1-1c98-44d5-b453-70f900533b89 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bacd4ab7-51ad-4943-9d16-92bd1fc1adfa", "name": "network-vif-deleted", "tag": "299443a4-87d4-4257-8710-d074d71d9851"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:32.747791 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-14846c21-9af3-4c3a-a569-e539eaa873a9 req-c9bd9cf1-1c98-44d5-b453-70f900533b89 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:32.750895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-14846c21-9af3-4c3a-a569-e539eaa873a9 req-c9bd9cf1-1c98-44d5-b453-70f900533b89 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:32.751336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-14846c21-9af3-4c3a-a569-e539eaa873a9 req-c9bd9cf1-1c98-44d5-b453-70f900533b89 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:32.799827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-14846c21-9af3-4c3a-a569-e539eaa873a9 req-c9bd9cf1-1c98-44d5-b453-70f900533b89 service nova] Creating event network-vif-deleted:299443a4-87d4-4257-8710-d074d71d9851 for instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa on hv-cinder-84914 Oct 12 02:35:32.813918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-14846c21-9af3-4c3a-a569-e539eaa873a9 req-c9bd9cf1-1c98-44d5-b453-70f900533b89 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109449 Oct 12 02:35:32.814979 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 445/885] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:32.823578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:32.841923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:32.841923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:32.841923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-23d57c11-483f-4997-873d-8808701a8f89 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.284277 Oct 12 02:35:32.866150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 441/886] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:32 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:33.502988 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:33.504409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:33.519577 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:33.520375 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:33.520836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:33.521706 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:33.522386 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:33.523039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:33.575900 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:33.576874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:33.577540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:33.578048 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lazy-loading 'fault' on Instance uuid bacd4ab7-51ad-4943-9d16-92bd1fc1adfa {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:35:33.653289 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '945de18174894cd683ebf34372e845f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c57365923b844fe9b29d9e223ad49ce4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:33.671498 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cf93e5eb-d310-45bc-8d47-a6592a13bcc6 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 200 len: 1348 microversion: 2.1 time: 0.173138 Oct 12 02:35:33.672068 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 446/887] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:33 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 1348 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:33.710921 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:33.713224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:33.730942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:33.731448 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:33.731888 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:33.734037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:33.734510 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:33.734939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:33.873145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:33.873773 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:33.897221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:33.904842 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:33.905878 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:33.906803 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:33.908246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:33.908929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:34.055182 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:34.070099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d0006282-8013-49a5-a556-cd0a47d29b27 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1557 microversion: 2.1 time: 0.370311 Oct 12 02:35:34.084741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 442/888] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:33 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1557 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:35:34.095417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:34.095927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:34.103647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:34.104592 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:34.105727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:34.106658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:34.107722 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:34.108303 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:34.220305 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8f739af9-46b9-4650-9265-3fa2703f446d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.129399 Oct 12 02:35:34.221542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 443/889] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:35:34 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-interface => generated 284 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:34.592281 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:34.626201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:34.637449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:34.643763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-87c3972a-6f31-42d4-bd6c-63751108af98 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.775039 Oct 12 02:35:34.657281 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 447/890] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:33 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 784 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:34.697712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:34.698337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] [instance: bacd4ab7-51ad-4943-9d16-92bd1fc1adfa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:34.712945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:34.714442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:34.715701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:34.717705 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:34.719687 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:34.721278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:34.774545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] HTTP exception thrown: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa could not be found. Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Request method failure captured: Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa HTTP/1.1 Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRidSyUKq-WdBdCxWbgglikMhhfYz1_750aRJ3JNLnVpQI2EhxeySVDD56d5zYyPS1k5psNp39aBg5ZtgARA2ju8S3ab1i23JMhVJihs9X8VwifkKjjY_4Dd9VHyHWiqbCU71MOBNizxb-0A-z6bcsNz-kBV7GgNN0EbgHWavtE9JBiY Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: c57365923b844fe9b29d9e223ad49ce4 Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-TestMinimumBasicScenario-608340467 Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:35:34.778040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c57365923b844fe9b29d9e223ad49ce4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c57365923b844fe9b29d9e223ad49ce4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c57365923b844fe9b29d9e223ad49ce4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c57365923b844fe9b29d9e223ad49ce4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-TestMinimumBasicScenario-608340467 Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: c57365923b844fe9b29d9e223ad49ce4 Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-TestMinimumBasicScenario-608340467 Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-TestMinimumBasicScenario-608340467-project Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 945de18174894cd683ebf34372e845f0 Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-TestMinimumBasicScenario-608340467-project Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa could not be found. Oct 12 02:35:34.779585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'bacd4ab7-51ad-4943-9d16-92bd1fc1adfa'} Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa could not be found. Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:35:34.780670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa could not be found. Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa could not be found. Oct 12 02:35:34.782223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:35:34.784358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Returning 404 to user: Instance bacd4ab7-51ad-4943-9d16-92bd1fc1adfa could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:35:34.786112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c4f4f002-3506-41b6-8b19-b77f0fe5d8e7 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa" status: 404 len: 111 microversion: 2.1 time: 0.093114 Oct 12 02:35:34.790342 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 444/891] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:35:34 2022] GET /compute/v2.1/servers/bacd4ab7-51ad-4943-9d16-92bd1fc1adfa => generated 111 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:35:34.796188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-941a9ab1-27c8-4bd2-887a-c4fde4097ca3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] Calling method '.version_select at 0x7f59d33beaf0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:34.847872 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-941a9ab1-27c8-4bd2-887a-c4fde4097ca3 tempest-TestMinimumBasicScenario-608340467 tempest-TestMinimumBasicScenario-608340467-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1657289250" status: 202 len: 0 microversion: 2.1 time: 0.055700 Oct 12 02:35:34.853625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 448/892] 10.222.0.23 () {58 vars in 1272 bytes} [Wed Oct 12 02:35:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1657289250 => generated 0 bytes in 59 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 12 02:35:35.675416 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:35.676166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:35.684784 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:35.685569 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:35.686839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:35.689638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:35.690312 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:35.693420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:36.056525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-b54685fd-7ca1-408e-b1d2-eff4cf866677 req-4169e36a-18a0-4e9b-ac0d-0f1fd9116cd4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b142a74-f235-4ea2-b553-6a0e8bac1c91", "tag": "93c61a39-b587-4e1c-9122-a17862f7c444"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:36.079611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-b54685fd-7ca1-408e-b1d2-eff4cf866677 req-4169e36a-18a0-4e9b-ac0d-0f1fd9116cd4 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:36.080283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-b54685fd-7ca1-408e-b1d2-eff4cf866677 req-4169e36a-18a0-4e9b-ac0d-0f1fd9116cd4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:36.080764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-b54685fd-7ca1-408e-b1d2-eff4cf866677 req-4169e36a-18a0-4e9b-ac0d-0f1fd9116cd4 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:36.082351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:36.091260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=100 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:36.105712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:36.109808 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:36.111299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-49b495b9-8b9e-4163-8928-8004a94348c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.440094 Oct 12 02:35:36.113060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-b54685fd-7ca1-408e-b1d2-eff4cf866677 req-4169e36a-18a0-4e9b-ac0d-0f1fd9116cd4 service nova] Creating event network-changed:93c61a39-b587-4e1c-9122-a17862f7c444 for instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 on hv-cinder-84914 Oct 12 02:35:36.119680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 445/893] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:35 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:36.125263 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-b54685fd-7ca1-408e-b1d2-eff4cf866677 req-4169e36a-18a0-4e9b-ac0d-0f1fd9116cd4 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071143 Oct 12 02:35:36.126081 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 449/894] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:35:37.147822 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:37.148590 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:37.161627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:37.164367 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:37.165223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:37.168807 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:37.169655 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:37.170348 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:37.459024 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:37.470954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:37.473822 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:37.475154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8e1d5f86-adc2-4daa-9d7a-b3bc33cb0589 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.333453 Oct 12 02:35:37.481617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 446/895] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:37 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:38.504740 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:38.506101 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:38.558316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:38.558893 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:38.559361 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:38.560120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:38.560597 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:38.561054 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:38.853552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:38.865324 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:38.869125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:38.870614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fe582db4-bffe-4b61-93ef-bb083e2d3576 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.371222 Oct 12 02:35:38.885632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 450/896] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:38 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:39.900229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:39.901052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:39.908150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:39.908821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:39.909363 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:39.910305 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:39.910870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:39.911395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:40.178044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:40.192784 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:40.195221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:40.197077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e04f60ca-e93f-46f6-9e26-9f6eaba530c3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.301352 Oct 12 02:35:40.205012 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 447/897] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:39 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:41.223795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:41.224563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:41.235410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:41.237122 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:41.240059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:41.241156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:41.241859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:41.244678 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:41.429632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:41.442269 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:41.450211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:41.452292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2305a875-cecc-4d15-bbbb-8e79e85f2ed6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.232655 Oct 12 02:35:41.454615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 451/898] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:41 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:42.494569 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:42.495429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:42.507405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:42.507989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:42.508662 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:42.509855 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:42.510942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:42.511752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:42.750324 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:42.760895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:42.762500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:42.764079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a70359c3-61b3-4cca-b18d-c34ed4d41d6a tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1737 microversion: 2.20 time: 0.274531 Oct 12 02:35:42.764819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 448/899] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:42 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1737 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:43.790692 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:43.791389 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:43.797760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:43.798247 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:43.798682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:43.799373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:43.800007 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:43.800436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:43.997317 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:44.006741 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:44.008177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:44.009906 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3a899234-b935-467f-a651-aa4aabd53ee3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1719 microversion: 2.20 time: 0.223265 Oct 12 02:35:44.010464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 452/900] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:43 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1719 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:45.036204 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:45.036838 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:45.044273 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:45.044959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:45.045394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:45.046154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:45.046601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:45.047213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:45.052363 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-92d633d9-b0aa-4bd6-8c57-45a7f3efc311 req-eb9c850c-a4b4-42f8-bfb1-3dad3f51144c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a61f7314-595f-4a3e-9288-0d25df1b9607", "name": "network-vif-unplugged", "status": "completed", "tag": "fe5a040d-6b9b-4b90-a3c9-76a1085a756e"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:45.066934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-92d633d9-b0aa-4bd6-8c57-45a7f3efc311 req-eb9c850c-a4b4-42f8-bfb1-3dad3f51144c service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:45.068004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-92d633d9-b0aa-4bd6-8c57-45a7f3efc311 req-eb9c850c-a4b4-42f8-bfb1-3dad3f51144c service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:45.068724 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-92d633d9-b0aa-4bd6-8c57-45a7f3efc311 req-eb9c850c-a4b4-42f8-bfb1-3dad3f51144c service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:45.080944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-92d633d9-b0aa-4bd6-8c57-45a7f3efc311 req-eb9c850c-a4b4-42f8-bfb1-3dad3f51144c service nova] Creating event network-vif-unplugged:fe5a040d-6b9b-4b90-a3c9-76a1085a756e for instance a61f7314-595f-4a3e-9288-0d25df1b9607 on hv-cinder-84914 Oct 12 02:35:45.086774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-92d633d9-b0aa-4bd6-8c57-45a7f3efc311 req-eb9c850c-a4b4-42f8-bfb1-3dad3f51144c service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.038551 Oct 12 02:35:45.087900 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 453/901] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:35:45 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:35:45.193549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:45.203444 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:45.204944 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:45.206136 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b4ef7b5c-c089-4365-a1d3-b2364975a154 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1719 microversion: 2.20 time: 0.174627 Oct 12 02:35:45.206752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 449/902] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:45 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1719 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:46.232593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:46.233235 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:46.240120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:46.240608 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:46.241064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:46.241769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:46.242322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:46.242741 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:46.408428 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:46.416612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:46.418083 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:46.419239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bf9cd8ed-0257-4c3d-8406-e090f6db268f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1662 microversion: 2.20 time: 0.191205 Oct 12 02:35:46.419799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 454/903] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:46 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1662 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:46.447163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0a0ba44c-c542-49f5-86e2-c02d99815b0a"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:46.449823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:46.456221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:46.456725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:46.457184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:46.457851 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:46.458461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:46.458882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:46.759396 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7bf39de9-ef86-4f22-8788-159999e906fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.316509 Oct 12 02:35:46.760724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 450/904] 10.222.0.23 () {62 vars in 1364 bytes} [Wed Oct 12 02:35:46 2022] POST /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-volume_attachments => generated 188 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 12 02:35:46.894282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:46.894904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:46.901427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:46.901987 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:46.902427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:46.903119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:46.903572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:46.904003 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:47.186419 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6666a945-3ffa-4f78-a23c-ee78c7f48834 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-volume_attachments/0a0ba44c-c542-49f5-86e2-c02d99815b0a" status: 202 len: 0 microversion: 2.20 time: 0.295991 Oct 12 02:35:47.187467 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 455/905] 10.222.0.23 () {60 vars in 1457 bytes} [Wed Oct 12 02:35:46 2022] DELETE /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-volume_attachments/0a0ba44c-c542-49f5-86e2-c02d99815b0a => generated 0 bytes in 298 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:47.254891 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:35:47.255556 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:47.263096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:47.263756 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:47.264285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:47.265910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:47.266686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:47.267337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:47.302941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118865) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 12 02:35:47.410843 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-73a0f8a9-cc91-44e8-bac1-a0ec0c43342d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "POST /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/action" status: 202 len: 0 microversion: 2.20 time: 0.160681 Oct 12 02:35:47.413197 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 451/906] 10.222.0.23 () {62 vars in 1319 bytes} [Wed Oct 12 02:35:47 2022] POST /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/action => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:35:47.422026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:47.423239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:47.438569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:47.439419 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:47.440102 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:47.441730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:47.443543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:47.444562 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:47.861862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:47.886030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:47.887674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:47.889246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fe0b5de4-03cb-4b95-9b10-0a1eb64e3860 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1670 microversion: 2.20 time: 0.471517 Oct 12 02:35:47.889923 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 456/907] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:47 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1670 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:48.920615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:48.921398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:48.935313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:48.936604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:48.938965 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:48.939754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:48.945615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:48.946930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:49.489651 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:49.499554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:49.501615 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:49.503830 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8433d233-661f-4385-925c-b6a380d1e1af tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.588674 Oct 12 02:35:49.505021 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 452/908] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:48 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:50.534638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:50.535829 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:50.542322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:35:50.543808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:50.544535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:50.545251 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:50.546341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:50.547019 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:50.556313 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:51.341566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:51.350501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:51.359495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:51.360909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4bcb352b-ea1e-4201-9f33-397d47e7aea1 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.833315 Oct 12 02:35:51.361960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 457/909] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:50 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:52.395058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:52.395742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:52.410544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:52.411260 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:52.411911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:52.412942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:52.413616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:52.414103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:52.684544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:52.695266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:52.697632 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:52.699179 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-035ccad0-286f-4776-b15a-88e181d5ef7d tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.314825 Oct 12 02:35:52.700314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 453/910] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:52 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:53.732812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:53.733518 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:53.750555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:53.751130 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:53.751484 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:53.752183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:53.752633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:53.753078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:54.008712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:54.022216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:54.026680 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:54.029978 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6c6405fe-10b8-4ded-a6c5-18376ca30146 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.301884 Oct 12 02:35:54.031293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 458/911] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:53 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:55.058427 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:55.059565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:55.067832 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:55.073978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:55.074728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:55.075697 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:55.076362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:55.077022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:55.751099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:55.775300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:55.777177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:55.781073 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-73e433fc-ff90-4a7a-9fd8-ce047cec5270 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.727073 Oct 12 02:35:55.800547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 454/912] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:55 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 742 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:56.815186 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:56.816681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:56.828765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:56.829545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:56.830266 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:56.831146 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:56.833590 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:56.834193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:57.379111 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:57.395864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:57.397621 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:57.403254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-21f9531f-4330-4ea1-b1d9-6de04cace9fb tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.588305 Oct 12 02:35:57.404070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 459/913] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:56 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:58.437364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:58.438098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:58.453607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:58.458522 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:58.459473 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:58.463046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:58.465928 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:58.466699 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:58.885764 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:58.894797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:58.909152 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:35:58.915085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-11ca8a47-0dc2-4beb-8261-77cd206e41b7 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.482224 Oct 12 02:35:58.927165 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 455/914] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:58 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:35:59.946261 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:35:59.946862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:35:59.956171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:59.957047 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:59.957689 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:59.958607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:35:59.959330 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:35:59.959944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:35:59.999359 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:00.242183 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:00.267148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:00.273522 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:00.275527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a1a54a9b-7533-4afd-b571-ee2622993098 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.334043 Oct 12 02:36:00.290488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 460/915] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:35:59 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:00.977603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8ea27fd9-8a23-458c-aa97-f4d8fc011728 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Action: 'create', calling method: .version_select at 0x7f59d34adb80>, body: {"keypair": {"name": "tempest-TestShelveInstance-474987065", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCp0RGFjpF3p4+Q8khVqhq8WCFDZO6EW2SqOJw4lQX6hBDRkaW7R8FWKVwEeYSqUEbtJdnS63JqNB/Rmo/zHWHy7ub8cE1XatFF3LTyUnGP4+uJOQmW6EQH0JxWAutqXJQ=="}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:00.984363 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:01.002461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-8ea27fd9-8a23-458c-aa97-f4d8fc011728 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Getting quotas for project b1601a3e4f41405d8e9001af6da86b46. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:36:01.006877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-8ea27fd9-8a23-458c-aa97-f4d8fc011728 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Getting quotas for user 60e6b5fcfd9d454bb10dd314bcdbedf6 and project b1601a3e4f41405d8e9001af6da86b46. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:36:01.031971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8ea27fd9-8a23-458c-aa97-f4d8fc011728 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.400537 Oct 12 02:36:01.032913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 456/916] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:36:00 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:01.306623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:01.307848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:01.314934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:01.315427 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:01.316048 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:01.316795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:01.317255 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:01.317862 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:01.495339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:01.503900 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:01.505387 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:01.506569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-84f12d82-de2c-4427-abba-b197fb69c840 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.203875 Oct 12 02:36:01.512420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 461/917] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:01 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:02.542410 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:02.543311 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:02.562269 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:02.562821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:02.563104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:02.563814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:02.564266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:02.564688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:02.775362 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:02.783772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:02.785450 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:02.786614 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-09a2a207-33e1-4794-9bf5-080426f01cba tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.250327 Oct 12 02:36:02.787249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 457/918] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:02 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:03.813931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:03.814769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:03.823423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:03.824227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:03.824882 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:03.825876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:03.826333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:03.827136 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:03.984613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:03.996639 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:04.001233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:04.002488 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c5fa291d-d82c-4ea7-aacb-97f8e5b4603c tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.192914 Oct 12 02:36:04.002976 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 462/919] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:03 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:05.030138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:05.030751 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:05.042579 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:05.043534 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:05.044357 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:05.045347 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:05.046043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:05.046716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:05.228113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:05.237597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:05.239916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:05.242252 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c1f614ed-db06-4ee7-b955-da109e9b7c91 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.215613 Oct 12 02:36:05.243334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 458/920] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:05 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:05.283484 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-8cc9b33b-fcd0-4ffa-927e-d8113d82e309 req-5120839f-b940-45d6-a5d3-35687452b036 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a61f7314-595f-4a3e-9288-0d25df1b9607", "name": "network-vif-plugged", "status": "completed", "tag": "fe5a040d-6b9b-4b90-a3c9-76a1085a756e"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:05.297847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-8cc9b33b-fcd0-4ffa-927e-d8113d82e309 req-5120839f-b940-45d6-a5d3-35687452b036 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:05.298641 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-8cc9b33b-fcd0-4ffa-927e-d8113d82e309 req-5120839f-b940-45d6-a5d3-35687452b036 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:05.299298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-8cc9b33b-fcd0-4ffa-927e-d8113d82e309 req-5120839f-b940-45d6-a5d3-35687452b036 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:05.312861 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-8cc9b33b-fcd0-4ffa-927e-d8113d82e309 req-5120839f-b940-45d6-a5d3-35687452b036 service nova] Creating event network-vif-plugged:fe5a040d-6b9b-4b90-a3c9-76a1085a756e for instance a61f7314-595f-4a3e-9288-0d25df1b9607 on hv-cinder-84914 Oct 12 02:36:05.319503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-8cc9b33b-fcd0-4ffa-927e-d8113d82e309 req-5120839f-b940-45d6-a5d3-35687452b036 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040674 Oct 12 02:36:05.320824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 463/921] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:36:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:06.269760 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:06.273795 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:06.277736 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:06.278222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:06.278816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:06.279484 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:06.279959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:06.280420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:06.414969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:06.418480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:06.424263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:06.425715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:06.426857 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-62b3992f-a540-4551-bda3-930465bd2e30 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.161121 Oct 12 02:36:06.427406 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 459/922] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:06 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:07.454489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:07.455399 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:07.463382 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:07.463973 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:07.464471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:07.465892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:07.466464 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:07.467149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:07.597641 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:07.606271 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:07.608009 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:07.609189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ed2fef9b-12c3-4e57-b07d-dcc1079c01a3 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1728 microversion: 2.20 time: 0.159311 Oct 12 02:36:07.609751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 464/923] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:07 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1728 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:08.232183 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:08.232788 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:08.239468 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:08.239996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:08.240424 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:08.241135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:08.241585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:08.242003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:08.267728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b66b4136-8bf2-48ec-94c3-dee0cbadb48d tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/ips" status: 200 len: 150 microversion: 2.1 time: 0.040067 Oct 12 02:36:08.268215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 460/924] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:36:08 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/ips => generated 150 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:08.634678 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:08.635874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:08.642877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:08.643366 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:08.643794 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:08.644485 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:08.645135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:08.645551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:08.774039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:08.782562 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:08.784035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:08.785219 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-019fb0ae-9cce-4102-94c4-f5449e73bd72 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1715 microversion: 2.20 time: 0.154423 Oct 12 02:36:08.785775 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 465/925] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:08 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1715 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:09.615599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "edbd2eb3-8d37-44af-8627-25cbf9de4a48"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:09.618013 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:09.625056 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:09.625539 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:09.625969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:09.626656 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:09.627107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:09.627523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:09.908145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e89c5679-6ddf-4782-8ef7-ac061f8bb809 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "POST /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.296019 Oct 12 02:36:09.908780 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 461/926] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:36:09 2022] POST /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments => generated 194 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:17.594393 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:17.594917 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:17.601631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:17.602106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:17.602566 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:17.603227 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:17.603672 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:17.604093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:17.688454 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'services' on Instance uuid 4b142a74-f235-4ea2-b553-6a0e8bac1c91 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:36:17.763165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b6e4d351-5be2-44f7-ba9f-e38307124b6b tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments/edbd2eb3-8d37-44af-8627-25cbf9de4a48" status: 202 len: 0 microversion: 2.1 time: 0.172392 Oct 12 02:36:17.763682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 466/927] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:36:17 2022] DELETE /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments/edbd2eb3-8d37-44af-8627-25cbf9de4a48 => generated 0 bytes in 174 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:36:17.775477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:17.779465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:17.782577 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=4 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:17.789895 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:17.790383 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:17.791022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:17.791729 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:17.792185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:17.792642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:17.845299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-205f3133-55bb-4098-91c7-a812fde42aba tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075714 Oct 12 02:36:17.845927 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 462/928] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:36:17 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:18.654302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-2137482952", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-474987065", "security_groups": [{"name": "tempest-secgroup-smoke-1488987721"}], "networks": [{"uuid": "3579f488-1360-4e5f-a1d3-8c57917ab513"}], "block_device_mapping_v2": [{"uuid": "54fee5f7-ea80-41bc-b7fd-4d000883370d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:18.783420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] validate_networks() for [('3579f488-1360-4e5f-a1d3-8c57917ab513', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:36:18.859889 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:18.863025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:18.869483 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:18.870055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:18.870597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:18.871358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:18.872035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:18.872526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:18.910787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0de914d9-d4e4-4748-a89c-da40999bafbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055747 Oct 12 02:36:18.911488 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 463/929] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:36:18 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:36:18.978724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:18.979224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:18.985344 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:18.986167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:18.986601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:18.987285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:18.987915 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:18.988339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:19.014433 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:36:19.103130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b97dcff7-1062-47a5-b0e5-4feb2ef97abf tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 204 len: 0 microversion: 2.1 time: 0.127772 Oct 12 02:36:19.104343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 464/930] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:36:18 2022] DELETE /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 12 02:36:19.116346 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:19.117156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:19.126480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:19.127042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:19.127612 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:19.128611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:19.129084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:19.129594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:19.210712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:36:19.211345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:36:19.212109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:36:19.212856 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:36:19.213469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:36:19.224810 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:19.225302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:19.226179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:19.226921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:19.227480 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:19.227915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:19.247217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Getting quotas for project b1601a3e4f41405d8e9001af6da86b46. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:36:19.251307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Getting quotas for user 60e6b5fcfd9d454bb10dd314bcdbedf6 and project b1601a3e4f41405d8e9001af6da86b46. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:36:19.267459 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:19.277391 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-22bd6474-38b4-4344-a43a-dea61f1f6ec9 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1682 microversion: 2.1 time: 0.165910 Oct 12 02:36:19.282454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 465/931] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:19 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1682 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:19.332672 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:36:19.334956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='54fee5f7-ea80-41bc-b7fd-4d000883370d',volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:36:19.470099 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ae59ddcf-d71d-4997-aa51-af0410e36943 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.820323 Oct 12 02:36:19.471228 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 467/932] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:36:18 2022] POST /compute/v2.1/servers => generated 406 bytes in 823 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:36:19.483425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-94045ac1-1f27-4c9d-8ef4-db393aae150b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:19.484046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-94045ac1-1f27-4c9d-8ef4-db393aae150b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:19.550808 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-94045ac1-1f27-4c9d-8ef4-db393aae150b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:19.557688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-94045ac1-1f27-4c9d-8ef4-db393aae150b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1088 microversion: 2.1 time: 0.079062 Oct 12 02:36:19.558886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 466/933] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:19 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1088 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:20.302960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:20.304213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:20.311191 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:20.311669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:20.312098 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:20.312816 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:20.313270 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:20.313687 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:20.440921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:20.449059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b9d4efb5-11dd-4478-885e-bf1af928f17c tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1682 microversion: 2.1 time: 0.149721 Oct 12 02:36:20.449631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 468/934] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:20 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1682 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:20.580487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:20.581143 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:20.588098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:20.588743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:20.589315 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:20.590063 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:20.590681 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:20.591184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:20.657381 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:20.664981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b8b63298-70c8-495f-8f9e-bc5bb6cdc597 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1138 microversion: 2.1 time: 0.088371 Oct 12 02:36:20.665542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 467/935] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:20 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1138 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:21.473418 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:21.474033 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:21.477408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:21.494459 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:21.494960 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:21.495398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:21.496122 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:21.496575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:21.497022 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:21.694267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:21.695445 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:21.716168 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:21.716646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:21.717304 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:21.717996 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:21.718444 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:21.718858 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:21.752463 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:21.766264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-738e1ca1-7132-42eb-8069-f271bd5f4cf6 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1682 microversion: 2.1 time: 0.296576 Oct 12 02:36:21.767439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 469/936] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:21 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1682 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:21.806331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:21.817104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8ad6e309-4771-431e-9feb-692c8e560f78 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1204 microversion: 2.1 time: 0.132785 Oct 12 02:36:21.832065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 468/937] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:21 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1204 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:22.138149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-5e3a0b08-f184-4e55-852d-be80b5d16492 req-319cfe7c-6189-46b3-b85a-0352431efa8f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b142a74-f235-4ea2-b553-6a0e8bac1c91", "name": "network-vif-deleted", "tag": "93c61a39-b587-4e1c-9122-a17862f7c444"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:22.149779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5e3a0b08-f184-4e55-852d-be80b5d16492 req-319cfe7c-6189-46b3-b85a-0352431efa8f service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:22.150533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5e3a0b08-f184-4e55-852d-be80b5d16492 req-319cfe7c-6189-46b3-b85a-0352431efa8f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:22.151139 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5e3a0b08-f184-4e55-852d-be80b5d16492 req-319cfe7c-6189-46b3-b85a-0352431efa8f service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:22.162920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-5e3a0b08-f184-4e55-852d-be80b5d16492 req-319cfe7c-6189-46b3-b85a-0352431efa8f service nova] Creating event network-vif-deleted:93c61a39-b587-4e1c-9122-a17862f7c444 for instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 on hv-cinder-84914 Oct 12 02:36:22.169509 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-5e3a0b08-f184-4e55-852d-be80b5d16492 req-319cfe7c-6189-46b3-b85a-0352431efa8f service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035776 Oct 12 02:36:22.170934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 470/938] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:36:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:22.790365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:22.791900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:22.801563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:22.802786 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:22.803733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:22.804993 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:22.806438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:22.807376 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:22.846983 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:22.848429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:22.850899 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:22.851464 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:22.851962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:22.852507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lazy-loading 'fault' on Instance uuid 4b142a74-f235-4ea2-b553-6a0e8bac1c91 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:36:22.860049 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:22.866457 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:22.867576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:22.868523 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:22.869208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:22.869907 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:22.918865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1fceec891ef24ba3a39da72c90fb733a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '76e921b067f04677b76c245b5f93e112', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:22.933059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b2d90fcb-b159-425d-b20e-749e570f60c4 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 200 len: 1331 microversion: 2.1 time: 0.146672 Oct 12 02:36:22.933912 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 469/939] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:22 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 1331 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:23.262386 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:23.271209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1087e1a8-1945-4daf-9fc3-c01a0d711feb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.430629 Oct 12 02:36:23.271912 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 471/940] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:22 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:36:23.594085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-c9b4ec41-69a8-4b0d-8325-8c3dbde6777e req-7c94176e-1e5e-41b5-9d9c-4ac4760808dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3d484497-9de8-43e6-b71c-d3a2d2488db8", "tag": "66a8b57d-c6be-4a74-91b2-545572f47798"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:23.607283 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c9b4ec41-69a8-4b0d-8325-8c3dbde6777e req-7c94176e-1e5e-41b5-9d9c-4ac4760808dc service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:23.608200 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c9b4ec41-69a8-4b0d-8325-8c3dbde6777e req-7c94176e-1e5e-41b5-9d9c-4ac4760808dc service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:23.608827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-c9b4ec41-69a8-4b0d-8325-8c3dbde6777e req-7c94176e-1e5e-41b5-9d9c-4ac4760808dc service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:23.620670 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-c9b4ec41-69a8-4b0d-8325-8c3dbde6777e req-7c94176e-1e5e-41b5-9d9c-4ac4760808dc service nova] Creating event network-changed:66a8b57d-c6be-4a74-91b2-545572f47798 for instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 on hv-cinder-84914 Oct 12 02:36:23.625237 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-c9b4ec41-69a8-4b0d-8325-8c3dbde6777e req-7c94176e-1e5e-41b5-9d9c-4ac4760808dc service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035072 Oct 12 02:36:23.625828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 470/941] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:36:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:23.959154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:23.959772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:23.966549 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:23.967040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:23.967476 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:23.968177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:23.968634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:23.969094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:23.983280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 HTTP/1.1 Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:36:23.984970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:23.986649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '4b142a74-f235-4ea2-b553-6a0e8bac1c91'} Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:23.987584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:23.989185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:23.990654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:23.990654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-130ad74f-c490-4184-b872-d743da759194 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 404 len: 111 microversion: 2.1 time: 0.031235 Oct 12 02:36:23.990654 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 472/942] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:23 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:36:23.997334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b46526d8-33fd-40e7-b292-fbf7a2dd0073 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '.version_select at 0x7f59d34f9940>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:24.017338 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b46526d8-33fd-40e7-b292-fbf7a2dd0073 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1738962876" status: 202 len: 0 microversion: 2.1 time: 0.025206 Oct 12 02:36:24.018029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 471/943] 10.222.0.23 () {58 vars in 1221 bytes} [Wed Oct 12 02:36:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1738962876 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:36:24.296016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:24.296638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:24.302721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:24.303193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:24.303632 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:24.309412 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:24.310059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:24.310504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:24.612947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:24.621286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a2289c0d-243a-42de-8da4-c2d3a72b725c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.329615 Oct 12 02:36:24.621980 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 473/944] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:24 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:25.644726 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:25.645455 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:25.652030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:25.652501 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:25.652954 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:25.653624 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:25.654066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:25.654480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:25.790869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:25.799224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-166abb6f-e796-4834-bc03-48c7158a893f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.158923 Oct 12 02:36:25.799786 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 472/945] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:25 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:26.135857 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:26.136341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:26.142703 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:26.143185 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:26.143615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:26.144298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:26.144776 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:26.145204 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:26.158742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 HTTP/1.1 Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:36:26.160252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.161892 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '4b142a74-f235-4ea2-b553-6a0e8bac1c91'} Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:26.162708 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.163925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.165180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:26.165180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2fa3699d-e012-4f16-b03a-df33e116b217 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 404 len: 111 microversion: 2.1 time: 0.029623 Oct 12 02:36:26.165180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 474/946] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:36:26 2022] DELETE /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:36:26.170535 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:26.171133 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 4b142a74-f235-4ea2-b553-6a0e8bac1c91] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:26.176961 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:26.177437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:26.177861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:26.178526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:26.178970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:26.179386 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:26.195723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 HTTP/1.1 Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:36:26.197641 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.199734 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '4b142a74-f235-4ea2-b553-6a0e8bac1c91'} Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:26.200630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. Oct 12 02:36:26.201906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:26.203015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 4b142a74-f235-4ea2-b553-6a0e8bac1c91 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:26.203015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-903df45d-4f65-44ee-8a11-6cab464670be tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91" status: 404 len: 111 microversion: 2.1 time: 0.033370 Oct 12 02:36:26.203015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 473/947] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:26 2022] GET /compute/v2.1/servers/4b142a74-f235-4ea2-b553-6a0e8bac1c91 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:36:26.822593 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:26.823791 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:26.829866 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:26.830353 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:26.830788 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:26.831477 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:26.831939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:26.832360 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:26.972347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:26.980560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-19470703-36df-4695-9382-66f145b268a1 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.162384 Oct 12 02:36:26.981400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 475/948] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:26 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:27.337387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:27.337867 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:27.344221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.344687 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.345141 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.345814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.346258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.346669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.360198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: DELETE /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 HTTP/1.1 Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:36:27.362003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.363648 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '928a91d5-600b-4707-9428-2e716f411885'} Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:27.365135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.367018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.368642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:27.368642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-107ff824-0119-43d9-834b-1ab22d049fbc tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 404 len: 111 microversion: 2.1 time: 0.029885 Oct 12 02:36:27.368642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 474/949] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:36:27 2022] DELETE /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:36:27.372926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:27.373529 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 928a91d5-600b-4707-9428-2e716f411885] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:27.379785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.380262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.380727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.381397 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.381841 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.382253 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.395884 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 HTTP/1.1 Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:36:27.397407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.398938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '928a91d5-600b-4707-9428-2e716f411885'} Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:27.399956 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. Oct 12 02:36:27.401402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.402812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 928a91d5-600b-4707-9428-2e716f411885 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:27.402812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-65108485-df81-478d-907b-d57f50d3a655 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885" status: 404 len: 111 microversion: 2.1 time: 0.030176 Oct 12 02:36:27.402812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 476/950] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:27 2022] GET /compute/v2.1/servers/928a91d5-600b-4707-9428-2e716f411885 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:36:27.407069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:27.407561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:27.413575 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.414061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.414492 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.415202 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.415696 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.416185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.431038 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: DELETE /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e HTTP/1.1 Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:36:27.432561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.434622 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '7dc118ab-0194-4cbb-b335-caf4c25cca7e'} Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:27.436098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.437688 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.439286 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:27.439286 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-91b66c50-d565-435a-88d2-06f2655fe871 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 404 len: 111 microversion: 2.1 time: 0.031245 Oct 12 02:36:27.439286 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 475/951] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:36:27 2022] DELETE /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:36:27.442988 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:27.443599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 7dc118ab-0194-4cbb-b335-caf4c25cca7e] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:27.450238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.450714 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.451145 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.451834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:27.452286 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:27.452718 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:27.465714 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e HTTP/1.1 Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:36:27.467209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.468914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '7dc118ab-0194-4cbb-b335-caf4c25cca7e'} Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:27.469999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. Oct 12 02:36:27.471402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:27.472860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 7dc118ab-0194-4cbb-b335-caf4c25cca7e could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:27.472860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5f9cad01-cec6-4f7e-a826-b1f3324b8580 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e" status: 404 len: 111 microversion: 2.1 time: 0.029396 Oct 12 02:36:27.472860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 477/952] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:27 2022] GET /compute/v2.1/servers/7dc118ab-0194-4cbb-b335-caf4c25cca7e => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:36:28.003831 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:28.004465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:28.010436 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:28.010913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:28.011340 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:28.012016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:28.012465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:28.013092 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:28.146421 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:28.154686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-db47b557-205e-4bcd-b25a-80bdc7db0acb tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.154598 Oct 12 02:36:28.155245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 476/953] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:27 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:28.609293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:28.609834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:28.616332 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:28.616836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:28.617267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:28.617947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:28.618426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:28.618839 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:28.633207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 HTTP/1.1 Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:36:28.634712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.636634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '76667a6b-65ab-4a9d-8af3-0440b0dad579'} Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:28.637533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.638961 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.640624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:28.640624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ea7aa9cf-69d6-41db-b146-74e94af04cbe tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "DELETE /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 404 len: 111 microversion: 2.1 time: 0.031024 Oct 12 02:36:28.640624 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 478/954] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:36:28 2022] DELETE /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:36:28.644968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:28.645698 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] [instance: 76667a6b-65ab-4a9d-8af3-0440b0dad579] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:28.654041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:28.654536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:28.654975 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:28.655683 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:28.656140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:28.656559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:28.670062 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] HTTP exception thrown: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Request method failure captured: Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 HTTP/1.1 Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaRDZNCu7KhGtLj-rvzjJHcw7RXmTEZAaouF_6nQXPRCF7I158J_tLqXqInWd8bwxAFOO8rVLHEnoJLrnE3qNwAcMpbHYdEwJGBHuiXRptpdjoBlRmWu2BJ6LyDptTjpRP8tIxfTLNIwOs8dUC5CD4pDc2OVW2EdFg_Iwe5m8w_zO4 Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:36:28.671584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/76e921b067f04677b76c245b5f93e112"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/76e921b067f04677b76c245b5f93e112"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_76e921b067f04677b76c245b5f93e112", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 76e921b067f04677b76c245b5f93e112 Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1675798980 Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 1fceec891ef24ba3a39da72c90fb733a Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeNegativeTest-1675798980-project Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.673140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '76667a6b-65ab-4a9d-8af3-0440b0dad579'} Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:36:28.674049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. Oct 12 02:36:28.675343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:36:28.676420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] Returning 404 to user: Instance 76667a6b-65ab-4a9d-8af3-0440b0dad579 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:36:28.676420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4084ca2a-6338-41c7-b834-18a7e357ab01 tempest-AttachVolumeNegativeTest-1675798980 tempest-AttachVolumeNegativeTest-1675798980-project] 10.222.0.23 "GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579" status: 404 len: 111 microversion: 2.1 time: 0.031850 Oct 12 02:36:28.676420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 477/955] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:28 2022] GET /compute/v2.1/servers/76667a6b-65ab-4a9d-8af3-0440b0dad579 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:36:29.178808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:29.179453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:29.186290 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:29.186769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:29.187198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:29.187876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:29.188325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:29.188769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:29.237203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:29.272603 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=6 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:29.419056 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:29.426807 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9052a273-fcda-40f1-b1a5-72d60591f843 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.252329 Oct 12 02:36:29.427365 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 479/956] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:29 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:30.450892 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:30.451557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:30.460009 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:30.465417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:30.465950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:30.466387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:30.467093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:30.467553 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:30.470594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:30.761419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:30.766024 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:30.771970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-cc4bf7f1-96cd-4080-8e0a-e2822d726de5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.325559 Oct 12 02:36:30.772541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 478/957] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:30 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:31.798589 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:31.800596 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:31.809555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:31.810610 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:31.811285 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:31.812553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:31.813415 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:31.814055 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:31.819420 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=89,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:32.056342 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:32.074573 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-cfbedb45-1ad8-46ab-9ca6-abf2ce1b1a85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.282427 Oct 12 02:36:32.092081 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 480/958] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:31 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:33.100178 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:33.100966 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:33.103326 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=1,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:33.110950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:33.111519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:33.112035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:33.112886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:33.113225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:33.113652 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:33.268145 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:33.278931 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-87b45ab0-2409-4997-8159-39636c22f2aa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.182972 Oct 12 02:36:33.298559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 479/959] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:33 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:33.381373 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-f135a486-c434-41a2-a0ea-c39235f0b7f8 req-2da6df24-719a-4095-9d20-d8d32185eeb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d484497-9de8-43e6-b71c-d3a2d2488db8", "name": "network-vif-plugged", "status": "completed", "tag": "66a8b57d-c6be-4a74-91b2-545572f47798"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:33.392898 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f135a486-c434-41a2-a0ea-c39235f0b7f8 req-2da6df24-719a-4095-9d20-d8d32185eeb8 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:33.393318 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f135a486-c434-41a2-a0ea-c39235f0b7f8 req-2da6df24-719a-4095-9d20-d8d32185eeb8 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:33.393945 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f135a486-c434-41a2-a0ea-c39235f0b7f8 req-2da6df24-719a-4095-9d20-d8d32185eeb8 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:33.408424 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-f135a486-c434-41a2-a0ea-c39235f0b7f8 req-2da6df24-719a-4095-9d20-d8d32185eeb8 service nova] Creating event network-vif-plugged:66a8b57d-c6be-4a74-91b2-545572f47798 for instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 on hv-cinder-84914 Oct 12 02:36:33.414256 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-f135a486-c434-41a2-a0ea-c39235f0b7f8 req-2da6df24-719a-4095-9d20-d8d32185eeb8 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037515 Oct 12 02:36:33.415051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 481/960] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:36:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:36:34.301962 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:34.302551 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:34.309507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:34.309959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:34.310392 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:34.311074 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:34.311525 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:34.311941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:34.442512 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:34.451922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-58c89de1-ff21-4000-9c88-5112fb4df643 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1260 microversion: 2.1 time: 0.153810 Oct 12 02:36:34.452669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 480/961] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:34 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:34.885431 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6ae14a30-f77d-4ab7-9aea-2e088e4df21e tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d36285e0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:34.931822 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6ae14a30-f77d-4ab7-9aea-2e088e4df21e tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/e6cc87c5-da44-426a-88cb-2d9f33795fa1" status: 200 len: 350 microversion: 2.1 time: 0.050942 Oct 12 02:36:34.932416 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 482/962] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:36:34 2022] GET /compute/v2.1/os-volumes/e6cc87c5-da44-426a-88cb-2d9f33795fa1 => generated 350 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:35.474623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:35.475230 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:35.482224 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:35.482921 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:35.483521 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:35.484388 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:35.484864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:35.485503 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:35.639337 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:35.647941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-59b6f2a6-5bda-4d2c-ba7a-620515237d10 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1443 microversion: 2.1 time: 0.177024 Oct 12 02:36:35.648435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 481/963] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:35 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1443 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:35.671340 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:35.672199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:35.679030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:35.679551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:35.679917 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:35.680603 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:35.681073 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:35.686236 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:35.838283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:35.845945 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2d6f9795-df90-4112-9fca-e8a8f74a4b85 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1443 microversion: 2.1 time: 0.179873 Oct 12 02:36:35.846495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 483/964] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:36:35 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1443 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:36.294989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-442ae8eb-37c7-45d1-bc88-2149604cff6f tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d357fca0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:36.342313 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-442ae8eb-37c7-45d1-bc88-2149604cff6f tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/eb48d001-3ff3-4f6f-8fcb-4927dc11eaee" status: 200 len: 351 microversion: 2.1 time: 0.051094 Oct 12 02:36:36.342914 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 482/965] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:36:36 2022] GET /compute/v2.1/os-volumes/eb48d001-3ff3-4f6f-8fcb-4927dc11eaee => generated 351 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:37.552078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-f3a04db4-4b36-4736-9bb0-2cc1aa367036 req-85ff5420-b329-4e5d-9cf0-7aece227c7d9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3d484497-9de8-43e6-b71c-d3a2d2488db8", "tag": "66a8b57d-c6be-4a74-91b2-545572f47798"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:36:37.561728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f3a04db4-4b36-4736-9bb0-2cc1aa367036 req-85ff5420-b329-4e5d-9cf0-7aece227c7d9 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:37.562211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f3a04db4-4b36-4736-9bb0-2cc1aa367036 req-85ff5420-b329-4e5d-9cf0-7aece227c7d9 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:37.562638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f3a04db4-4b36-4736-9bb0-2cc1aa367036 req-85ff5420-b329-4e5d-9cf0-7aece227c7d9 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:37.574030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-f3a04db4-4b36-4736-9bb0-2cc1aa367036 req-85ff5420-b329-4e5d-9cf0-7aece227c7d9 service nova] Creating event network-changed:66a8b57d-c6be-4a74-91b2-545572f47798 for instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 on hv-cinder-84914 Oct 12 02:36:37.578811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-f3a04db4-4b36-4736-9bb0-2cc1aa367036 req-85ff5420-b329-4e5d-9cf0-7aece227c7d9 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030395 Oct 12 02:36:37.579563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 484/966] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:36:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:40.856932 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c66f82c0-6163-47e0-8962-3240a8110585 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d3514280>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:40.914456 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c66f82c0-6163-47e0-8962-3240a8110585 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/431c782e-00f4-4676-8e2a-22b12fb606b5" status: 200 len: 351 microversion: 2.1 time: 0.063103 Oct 12 02:36:40.915029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 483/967] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 12 02:36:40 2022] GET /compute/v2.1/os-volumes/431c782e-00f4-4676-8e2a-22b12fb606b5 => generated 351 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:40.930053 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0a6c22d2-32c3-482e-b760-5f10e8f3cb00 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d36289d0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.014439 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0a6c22d2-32c3-482e-b760-5f10e8f3cb00 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.089214 Oct 12 02:36:41.015195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 485/968] 10.222.0.23 () {58 vars in 1134 bytes} [Wed Oct 12 02:36:40 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:41.030409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c4964bec-c20a-481f-9d70-e58c1deba700 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d34ad0d0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.114399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c4964bec-c20a-481f-9d70-e58c1deba700 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.088210 Oct 12 02:36:41.115299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 484/969] 10.222.0.23 () {58 vars in 1149 bytes} [Wed Oct 12 02:36:41 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:41.129704 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4d83b79d-f1f3-4833-b17b-54ba3d8b333e tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d3692a60>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.210843 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4d83b79d-f1f3-4833-b17b-54ba3d8b333e tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.084895 Oct 12 02:36:41.211614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 486/970] 10.222.0.23 () {58 vars in 1134 bytes} [Wed Oct 12 02:36:41 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:41.225496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-79fce599-e19b-4d84-8398-e16467265a1c tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d34ad670>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.458749 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-79fce599-e19b-4d84-8398-e16467265a1c tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.237307 Oct 12 02:36:41.459321 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 485/971] 10.222.0.23 () {58 vars in 1167 bytes} [Wed Oct 12 02:36:41 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:41.473132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c3cd36c3-d678-4f75-bbe8-ce5dcfdea63c tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d352cee0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.555424 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c3cd36c3-d678-4f75-bbe8-ce5dcfdea63c tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.086024 Oct 12 02:36:41.556004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 487/972] 10.222.0.23 () {58 vars in 1170 bytes} [Wed Oct 12 02:36:41 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:41.569776 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-93317d3d-c60c-4d1d-8d93-a037e3a15fcd tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d3506af0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.649634 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-93317d3d-c60c-4d1d-8d93-a037e3a15fcd tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.083629 Oct 12 02:36:41.650229 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 486/973] 10.222.0.23 () {58 vars in 1155 bytes} [Wed Oct 12 02:36:41 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:36:41.664044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fad1adce-f003-46fd-bb8b-88cfefc9adc5 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d36a30d0>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.746093 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fad1adce-f003-46fd-bb8b-88cfefc9adc5 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.085714 Oct 12 02:36:41.746671 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 488/974] 10.222.0.23 () {58 vars in 1188 bytes} [Wed Oct 12 02:36:41 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:36:41.760486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-92719f67-51f9-4850-bca1-82b1210e43a3 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] Calling method '.version_select at 0x7f59d3526040>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:41.842030 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-92719f67-51f9-4850-bca1-82b1210e43a3 tempest-VolumesTestJSON-836685522 tempest-VolumesTestJSON-836685522-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.085122 Oct 12 02:36:41.842588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 487/975] 10.222.0.23 () {58 vars in 1155 bytes} [Wed Oct 12 02:36:41 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:58.044820 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=29 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:58.045902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=15 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:58.050614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:58.052865 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:58.060034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:58.060521 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:58.060985 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:58.061692 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:58.062152 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:58.062574 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:58.091388 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d8b7f62a-f7b9-4bc6-a77a-1a1d23874ec6 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.048250 Oct 12 02:36:58.091935 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 489/976] 10.222.0.23 () {60 vars in 1343 bytes} [Wed Oct 12 02:36:58 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:36:58.152379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=5 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:58.153166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=18 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:36:58.157319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:58.157811 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:58.164465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:58.164971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:58.165408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:58.166093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:58.167149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:58.167675 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:58.191515 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:36:58.269080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d99b8629-0ccf-4c3c-9893-1bfac6d41b34 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 204 len: 0 microversion: 2.20 time: 0.118111 Oct 12 02:36:58.269685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 488/977] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 12 02:36:58 2022] DELETE /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 12 02:36:58.278647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:58.279276 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:58.285149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:58.285630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:58.286064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:58.286749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:58.287203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:58.287622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:58.414509 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:58.422524 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:58.423974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:58.425159 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-96bd7e27-679f-45f2-ba80-2865c9a6f9ed tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.151043 Oct 12 02:36:58.425712 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 490/978] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:58 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:36:59.450538 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:36:59.451330 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:36:59.457747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:59.458234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:59.458674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:59.459370 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:36:59.459821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:36:59.460246 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:36:59.611646 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:59.619732 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:59.621194 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:36:59.622339 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2ce62a5a-439a-4477-a15b-d1a685f4876f tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.176111 Oct 12 02:36:59.622896 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 489/979] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:36:59 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:37:00.651625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:00.651625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:00.662658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:00.663234 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:00.665035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:00.665918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:00.669110 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:00.669604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:00.946324 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:00.949352 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:00.950919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:00.952198 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-03f74d18-b49c-46e3-96df-7bb94fb5a979 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1721 microversion: 2.20 time: 0.308627 Oct 12 02:37:00.952951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 491/980] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:37:00 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1721 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:37:01.567824 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-fbcf4acd-ef29-4190-bb68-8d9cf6c88200 req-cdb7dcc7-47d3-4504-bb66-14db512e9c8d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a61f7314-595f-4a3e-9288-0d25df1b9607", "name": "network-vif-deleted", "tag": "fe5a040d-6b9b-4b90-a3c9-76a1085a756e"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:37:01.580477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-fbcf4acd-ef29-4190-bb68-8d9cf6c88200 req-cdb7dcc7-47d3-4504-bb66-14db512e9c8d service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:01.580987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-fbcf4acd-ef29-4190-bb68-8d9cf6c88200 req-cdb7dcc7-47d3-4504-bb66-14db512e9c8d service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:01.581408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-fbcf4acd-ef29-4190-bb68-8d9cf6c88200 req-cdb7dcc7-47d3-4504-bb66-14db512e9c8d service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:01.595266 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-fbcf4acd-ef29-4190-bb68-8d9cf6c88200 req-cdb7dcc7-47d3-4504-bb66-14db512e9c8d service nova] Creating event network-vif-deleted:fe5a040d-6b9b-4b90-a3c9-76a1085a756e for instance a61f7314-595f-4a3e-9288-0d25df1b9607 on hv-cinder-84914 Oct 12 02:37:01.603893 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-fbcf4acd-ef29-4190-bb68-8d9cf6c88200 req-cdb7dcc7-47d3-4504-bb66-14db512e9c8d service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039298 Oct 12 02:37:01.604431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 490/981] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:37:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:37:01.977944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:01.978767 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:01.986154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:01.991497 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:01.992600 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:01.993982 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:01.997100 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:01.997539 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:02.037399 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:02.038212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:02.038638 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:02.039131 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lazy-loading 'fault' on Instance uuid a61f7314-595f-4a3e-9288-0d25df1b9607 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:37:02.086793 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:02.124226 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:02.126669 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43f16ca13925411fb76a330b6c95395d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7f6eb3e59c464c749edb69e8370f54e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:02.129615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-29487c3e-ecac-4749-b97a-fa2819f997d2 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 200 len: 1370 microversion: 2.20 time: 0.154951 Oct 12 02:37:02.130781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 492/982] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:37:01 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 1370 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 12 02:37:03.158770 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:03.159587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:03.166131 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:03.166644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:03.167171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:03.167861 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:03.168308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:03.168747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:03.182821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] HTTP exception thrown: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Request method failure captured: Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 HTTP/1.1 Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaLTZf3hgmHzlK_C5LRX3NkyBEyTPu97mBszC48ey8TfKxHZ5Xwm1OSjbhNRDfu6az55cMVbT2b1UQJJRTOVbmicV1cVFK4d0Ia2uGNL8vRrUrQX61jMMFFyDtcjLIrxf5lvkeSXV-zHLvO6WZ4QYh1jwpdmF4zw11AKgY1DoAyrIo Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Openstack-Nova-Api-Version: 2.20 Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:37:03.184431 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_7f6eb3e59c464c749edb69e8370f54e5", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 43f16ca13925411fb76a330b6c95395d Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:03.186175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'a61f7314-595f-4a3e-9288-0d25df1b9607'} Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:37:03.187093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:03.188738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:03.190253 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Returning 404 to user: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:37:03.190253 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d9dde69d-1127-442c-a4e7-9d386a1e8d70 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 404 len: 111 microversion: 2.20 time: 0.031708 Oct 12 02:37:03.190253 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 491/983] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:37:03 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:37:03.195436 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ba2a177b-9946-44a0-bcbe-68ceca8f38c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '.version_select at 0x7f59d3693550>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:03.217535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ba2a177b-9946-44a0-bcbe-68ceca8f38c9 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-478946019" status: 204 len: 0 microversion: 2.20 time: 0.026043 Oct 12 02:37:03.221602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 493/984] 10.222.0.23 () {60 vars in 1259 bytes} [Wed Oct 12 02:37:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-478946019 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 12 02:37:06.083856 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:06.084360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:06.091691 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:06.092180 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:06.092617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:06.093359 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:06.093819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:06.094243 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:06.107847 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] HTTP exception thrown: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Request method failure captured: Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: DELETE /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 HTTP/1.1 Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaLTZf3hgmHzlK_C5LRX3NkyBEyTPu97mBszC48ey8TfKxHZ5Xwm1OSjbhNRDfu6az55cMVbT2b1UQJJRTOVbmicV1cVFK4d0Ia2uGNL8vRrUrQX61jMMFFyDtcjLIrxf5lvkeSXV-zHLvO6WZ4QYh1jwpdmF4zw11AKgY1DoAyrIo Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Openstack-Nova-Api-Version: 2.20 Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:37:06.109426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_7f6eb3e59c464c749edb69e8370f54e5", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 43f16ca13925411fb76a330b6c95395d Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.111025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'a61f7314-595f-4a3e-9288-0d25df1b9607'} Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:37:06.111841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.113085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.114290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Returning 404 to user: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:37:06.114290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8681087a-d999-4f1c-8d68-6792bb5bc98e tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 404 len: 111 microversion: 2.20 time: 0.030823 Oct 12 02:37:06.114290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 492/985] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 12 02:37:06 2022] DELETE /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:37:06.119952 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:06.120569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: a61f7314-595f-4a3e-9288-0d25df1b9607] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:06.129535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:06.130030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:06.130468 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:06.131168 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:06.131626 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:06.132052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:06.145756 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] HTTP exception thrown: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Request method failure captured: Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 HTTP/1.1 Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaLTZf3hgmHzlK_C5LRX3NkyBEyTPu97mBszC48ey8TfKxHZ5Xwm1OSjbhNRDfu6az55cMVbT2b1UQJJRTOVbmicV1cVFK4d0Ia2uGNL8vRrUrQX61jMMFFyDtcjLIrxf5lvkeSXV-zHLvO6WZ4QYh1jwpdmF4zw11AKgY1DoAyrIo Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Openstack-Nova-Api-Version: 2.20 Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:37:06.152825 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_7f6eb3e59c464c749edb69e8370f54e5", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 43f16ca13925411fb76a330b6c95395d Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.154394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'a61f7314-595f-4a3e-9288-0d25df1b9607'} Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:37:06.155264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. Oct 12 02:37:06.156587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:06.157729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Returning 404 to user: Instance a61f7314-595f-4a3e-9288-0d25df1b9607 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:37:06.157729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bf6a321c-f444-4aca-8d4b-1543f3297f63 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607" status: 404 len: 111 microversion: 2.20 time: 0.033455 Oct 12 02:37:06.157729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 494/986] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:37:06 2022] GET /compute/v2.1/servers/a61f7314-595f-4a3e-9288-0d25df1b9607 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:37:07.296271 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:07.296884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:07.303585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:07.304058 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:07.304485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:07.305192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:07.305638 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:07.306081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:07.318841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] HTTP exception thrown: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Request method failure captured: Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: DELETE /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa HTTP/1.1 Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRiaLTZf3hgmHzlK_C5LRX3NkyBEyTPu97mBszC48ey8TfKxHZ5Xwm1OSjbhNRDfu6az55cMVbT2b1UQJJRTOVbmicV1cVFK4d0Ia2uGNL8vRrUrQX61jMMFFyDtcjLIrxf5lvkeSXV-zHLvO6WZ4QYh1jwpdmF4zw11AKgY1DoAyrIo Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Openstack-Nova-Api-Version: 2.20 Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:37:07.320333 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_7f6eb3e59c464c749edb69e8370f54e5", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 43f16ca13925411fb76a330b6c95395d Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.322001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '48eb4bd4-2680-4fc6-85c0-dae85fb740fa'} Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:37:07.322762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.323978 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.325255 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Returning 404 to user: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:37:07.325255 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-038820e0-0ba4-45b3-973b-a3c4b9d5bdcf tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "DELETE /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 404 len: 111 microversion: 2.20 time: 0.029631 Oct 12 02:37:07.325255 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 493/987] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 12 02:37:07 2022] DELETE /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 12 02:37:07.330474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:07.331086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] [instance: 48eb4bd4-2680-4fc6-85c0-dae85fb740fa] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:07.337503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:07.338503 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:07.338935 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:07.339621 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:07.340073 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:07.340494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:07.354066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] HTTP exception thrown: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Request method failure captured: Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa HTTP/1.1 Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRiaLTZf3hgmHzlK_C5LRX3NkyBEyTPu97mBszC48ey8TfKxHZ5Xwm1OSjbhNRDfu6az55cMVbT2b1UQJJRTOVbmicV1cVFK4d0Ia2uGNL8vRrUrQX61jMMFFyDtcjLIrxf5lvkeSXV-zHLvO6WZ4QYh1jwpdmF4zw11AKgY1DoAyrIo Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Openstack-Nova-Api-Version: 2.20 Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:37:07.355915 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/7f6eb3e59c464c749edb69e8370f54e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_7f6eb3e59c464c749edb69e8370f54e5", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 7f6eb3e59c464c749edb69e8370f54e5 Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1953638504 Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: 43f16ca13925411fb76a330b6c95395d Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1953638504-project Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.357710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '48eb4bd4-2680-4fc6-85c0-dae85fb740fa'} Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:37:07.358514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. Oct 12 02:37:07.359754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:37:07.360871 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] Returning 404 to user: Instance 48eb4bd4-2680-4fc6-85c0-dae85fb740fa could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:37:07.360871 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-796bcaac-bc26-44a4-b33d-7ac46405c924 tempest-AttachVolumeShelveTestJSON-1953638504 tempest-AttachVolumeShelveTestJSON-1953638504-project] 10.222.0.23 "GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa" status: 404 len: 111 microversion: 2.20 time: 0.030744 Oct 12 02:37:07.360871 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 495/988] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 12 02:37:07 2022] GET /compute/v2.1/servers/48eb4bd4-2680-4fc6-85c0-dae85fb740fa => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 12 02:37:16.266107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9a91cce5-a487-4144-9fef-3e4baa8930c5 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'create', calling method: .version_select at 0x7f59d34f24c0>, body: {"keypair": {"name": "tempest-keypair-914788794", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEPMtMNUBe0OCA04pK2aY8lDo+Q/lHNg8DVZ9OpTAeMPzoCNzR5I8ODuyV67pG8l4pflHTyM1e1uLrTEsabntae9NXomo2UHZHYHef5mftXFK3wiuXcQO6cjrhKcBkYbog=="}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:37:16.285583 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-9a91cce5-a487-4144-9fef-3e4baa8930c5 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:37:16.289973 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-9a91cce5-a487-4144-9fef-3e4baa8930c5 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for user ea7aceaca15a47f89c674c237b0cacae and project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'key_pairs'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:37:16.317138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9a91cce5-a487-4144-9fef-3e4baa8930c5 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.240237 Oct 12 02:37:16.323967 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 494/989] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:37:16 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:37:18.161565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-189341465", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8484fb9c-5daa-4067-a233-ae010b930f21"}], "security_groups": [{"name": "tempest-securitygroup--363015057"}], "key_name": "tempest-keypair-914788794", "user_data": "}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:37:18.183338 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=17,nova_api:DELETE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:18.184039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=30 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:18.278628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] validate_networks() for [('8484fb9c-5daa-4067-a233-ae010b930f21', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:37:18.677837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:37:18.678254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:37:18.678722 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:37:18.679125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:37:18.679536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:37:18.689621 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:18.690109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:18.690543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:18.691462 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:18.691914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:18.692505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:18.709615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:37:18.713765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for user ea7aceaca15a47f89c674c237b0cacae and project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:37:18.726027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:37:18.729173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:37:18.781913 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-91e06dd0-c635-4749-9c4d-53cd05f03134 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.624670 Oct 12 02:37:18.788568 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 496/990] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:37:18 2022] POST /compute/v2.1/servers => generated 401 bytes in 626 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:37:18.792154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:18.796497 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a77cb901-6fb2-49ab-930d-4b2a198b144d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:18.797234 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a77cb901-6fb2-49ab-930d-4b2a198b144d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:18.882154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a77cb901-6fb2-49ab-930d-4b2a198b144d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:18.890742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a77cb901-6fb2-49ab-930d-4b2a198b144d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1236 microversion: 2.1 time: 0.100743 Oct 12 02:37:18.891485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 495/991] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:18 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1236 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:19.913731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:19.914369 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:19.920579 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:19.921443 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:19.922119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:19.923044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:19.923507 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:19.923938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:19.991719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:20.000230 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f860f8b8-4861-4b42-a6d0-f66b2480c087 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1288 microversion: 2.1 time: 0.090937 Oct 12 02:37:20.000822 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 497/992] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:19 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1288 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:21.029250 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:21.032823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:21.055010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:21.055660 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:21.056779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:21.058334 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:21.059620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:21.060335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:21.275663 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:21.284156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9cd3288c-2463-4ff0-b0bc-d48ca75b25d1 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1361 microversion: 2.1 time: 0.264534 Oct 12 02:37:21.286014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 496/993] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:21 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1361 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:21.945113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-5d40ed22-50cb-478e-8f46-42107a20611c req-34941f8f-a106-42e1-95a0-92cfb593d34e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5970385-f587-4e45-b2ca-9d5913f90396", "tag": "24ac17dd-767d-4f70-9eec-365ba24671e0"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:37:21.977276 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5d40ed22-50cb-478e-8f46-42107a20611c req-34941f8f-a106-42e1-95a0-92cfb593d34e service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:21.978681 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5d40ed22-50cb-478e-8f46-42107a20611c req-34941f8f-a106-42e1-95a0-92cfb593d34e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:21.979256 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5d40ed22-50cb-478e-8f46-42107a20611c req-34941f8f-a106-42e1-95a0-92cfb593d34e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:21.991789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-5d40ed22-50cb-478e-8f46-42107a20611c req-34941f8f-a106-42e1-95a0-92cfb593d34e service nova] Creating event network-changed:24ac17dd-767d-4f70-9eec-365ba24671e0 for instance f5970385-f587-4e45-b2ca-9d5913f90396 on hv-cinder-84914 Oct 12 02:37:21.998156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-5d40ed22-50cb-478e-8f46-42107a20611c req-34941f8f-a106-42e1-95a0-92cfb593d34e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063341 Oct 12 02:37:21.999325 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 498/994] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:37:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:37:22.309841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:22.311411 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:22.321166 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:22.321991 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:22.322909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:22.324137 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:22.325180 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:22.325882 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:22.512657 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:22.520441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1e5ff0d7-be0c-4ee4-b405-51d3236f74bd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1361 microversion: 2.1 time: 0.216004 Oct 12 02:37:22.521072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 497/995] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:22 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1361 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:23.545507 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:23.546243 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:23.553333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:23.554322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:23.555025 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:23.555881 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:23.556348 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:23.556799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:24.009411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:24.018553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d2d7e450-e5cb-4da5-bc16-cfc7c3a365b2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1361 microversion: 2.1 time: 0.478385 Oct 12 02:37:24.019272 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 499/996] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:23 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1361 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:25.040218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:25.040858 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:25.047144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:25.047689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:25.048043 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:25.048750 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:25.049440 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:25.049802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:25.181233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:25.189014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c7a266b3-6cc2-433f-9011-da9b16146a24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1361 microversion: 2.1 time: 0.152569 Oct 12 02:37:25.189864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 498/997] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:25 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1361 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:26.052155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-f032cf62-3293-48cc-9dc9-4c4458aba42f req-872bcfaf-49cc-459b-976c-5863bd962d97 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5970385-f587-4e45-b2ca-9d5913f90396", "name": "network-vif-plugged", "status": "completed", "tag": "24ac17dd-767d-4f70-9eec-365ba24671e0"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:37:26.065134 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f032cf62-3293-48cc-9dc9-4c4458aba42f req-872bcfaf-49cc-459b-976c-5863bd962d97 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:26.066558 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f032cf62-3293-48cc-9dc9-4c4458aba42f req-872bcfaf-49cc-459b-976c-5863bd962d97 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:26.067211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-f032cf62-3293-48cc-9dc9-4c4458aba42f req-872bcfaf-49cc-459b-976c-5863bd962d97 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:26.080283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-f032cf62-3293-48cc-9dc9-4c4458aba42f req-872bcfaf-49cc-459b-976c-5863bd962d97 service nova] Creating event network-vif-plugged:24ac17dd-767d-4f70-9eec-365ba24671e0 for instance f5970385-f587-4e45-b2ca-9d5913f90396 on hv-cinder-84914 Oct 12 02:37:26.086713 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-f032cf62-3293-48cc-9dc9-4c4458aba42f req-872bcfaf-49cc-459b-976c-5863bd962d97 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037827 Oct 12 02:37:26.088625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 500/998] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:37:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:37:26.214351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:26.215226 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:26.222701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:26.223186 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:26.224105 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:26.224842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:26.225297 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:26.225715 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:26.391258 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:26.403248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f56792dc-0935-4caf-a371-1c1739ca9008 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1361 microversion: 2.1 time: 0.194731 Oct 12 02:37:26.417331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 499/999] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:26 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1361 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:27.426945 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:27.428009 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:27.435034 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:27.435530 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:27.435970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:27.436682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:27.437190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:27.437780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:27.574506 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:27.582928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-21bb8518-a5f2-4a0c-96f1-218f50f9fc34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1361 microversion: 2.1 time: 0.160324 Oct 12 02:37:27.583581 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 501/1000] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:27 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1361 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:28.607869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:28.608475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:28.610761 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=41,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:28.615959 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:28.616441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:28.616897 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:28.617583 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:28.618616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:28.619041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:28.759983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:37:28.769504 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b48daeb4-a491-4f58-b605-96fff741bd24 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1547 microversion: 2.1 time: 0.165522 Oct 12 02:37:28.770296 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 500/1001] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:37:28 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1547 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:37:28.805133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:28.805133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:28.805133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:28.805133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=6 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:28.805133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:28.806084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:28.806084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:28.806084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:28.806084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:28.806408 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:28.895181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-96c7d7f4-6db7-404c-8c89-59d8e490b260 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.105242 Oct 12 02:37:28.895819 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 502/1002] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:37:28 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:37:29.832044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-8d077e1e-8a7d-42d8-9f72-68d481b07564 req-0758dc6d-867f-4cf7-90d9-6dc08c0382e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5970385-f587-4e45-b2ca-9d5913f90396", "tag": "24ac17dd-767d-4f70-9eec-365ba24671e0"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:37:29.837429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=6 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:29.843600 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8d077e1e-8a7d-42d8-9f72-68d481b07564 req-0758dc6d-867f-4cf7-90d9-6dc08c0382e1 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:29.844070 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8d077e1e-8a7d-42d8-9f72-68d481b07564 req-0758dc6d-867f-4cf7-90d9-6dc08c0382e1 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:29.844489 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-8d077e1e-8a7d-42d8-9f72-68d481b07564 req-0758dc6d-867f-4cf7-90d9-6dc08c0382e1 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:29.856292 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-8d077e1e-8a7d-42d8-9f72-68d481b07564 req-0758dc6d-867f-4cf7-90d9-6dc08c0382e1 service nova] Creating event network-changed:24ac17dd-767d-4f70-9eec-365ba24671e0 for instance f5970385-f587-4e45-b2ca-9d5913f90396 on hv-cinder-84914 Oct 12 02:37:29.861572 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-8d077e1e-8a7d-42d8-9f72-68d481b07564 req-0758dc6d-867f-4cf7-90d9-6dc08c0382e1 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033385 Oct 12 02:37:29.862174 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 501/1003] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:37:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:37:59.446038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:59.446891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=28 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:37:59.451009 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:37:59.451486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:37:59.458478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:59.458953 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:59.459381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:59.460059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:37:59.460510 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:37:59.460947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:37:59.485604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d0f642a1-daeb-4689-a355-b22a7567c175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/ips" status: 200 len: 145 microversion: 2.1 time: 0.040848 Oct 12 02:37:59.486151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 503/1004] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:37:59 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/ips => generated 145 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:38:03.629842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:03.630712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=32 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:03.636218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c3c69415-89ce-40eb-95d5-7db7e1759f03"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:38:03.638642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:03.645560 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:03.646042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:03.646475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:03.647154 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:03.647606 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:03.648025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:03.908085 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-94f82aee-8003-4842-8aed-6131cfce55e8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.279597 Oct 12 02:38:03.912484 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 502/1005] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:38:03 2022] POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-volume_attachments => generated 194 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:38:11.438752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0e692268-3d2e-47ca-b6ca-563b8f0d64c6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:38:11.441531 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:11.442191 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=4 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:11.447998 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e692268-3d2e-47ca-b6ca-563b8f0d64c6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:11.448484 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e692268-3d2e-47ca-b6ca-563b8f0d64c6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:11.448937 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0e692268-3d2e-47ca-b6ca-563b8f0d64c6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:11.508002 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0e692268-3d2e-47ca-b6ca-563b8f0d64c6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Going to try to stop instance {{(pid=118864) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 12 02:38:11.549451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0e692268-3d2e-47ca-b6ca-563b8f0d64c6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action" status: 202 len: 0 microversion: 2.1 time: 0.114493 Oct 12 02:38:11.550036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 504/1006] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:38:11 2022] POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:38:11.559185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:11.559866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:11.566010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:11.566487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:11.566995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:11.567691 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:11.568149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:11.568629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:11.697841 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:11.705896 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a1cf5177-9745-4a2b-b9a3-4b9e1eb7fbdd tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.151366 Oct 12 02:38:11.706449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 503/1007] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:11 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:12.731212 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:12.731909 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:12.738035 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:12.738519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:12.738953 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:12.739640 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:12.740099 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:12.740520 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:12.871635 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:12.879715 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fa761ac2-ecc6-4ee6-8f7b-d8fbba11fa1c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.153985 Oct 12 02:38:12.880268 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 505/1008] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:12 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:13.903179 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:13.903785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:13.910294 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:13.910781 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:13.911218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:13.911902 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:13.912361 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:13.912819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:14.042003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:14.050222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3051fc4a-24ff-4077-9c8d-623a8b0a4bd2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.151449 Oct 12 02:38:14.050779 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 504/1009] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:13 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:15.073299 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:15.073922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:15.080223 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:15.080727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:15.081168 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:15.081850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:15.082302 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:15.082723 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:15.208111 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:15.216684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5d70a5de-3fe0-46a7-a194-7c46d4932cdc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.147255 Oct 12 02:38:15.217281 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 506/1010] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:15 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:16.239904 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:16.240514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:16.247405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:16.247885 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:16.248341 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:16.249048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:16.249499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:16.249916 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:16.374923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:16.382945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c2e60a46-101a-475b-86cd-b72f67005f4a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.146947 Oct 12 02:38:16.383508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 505/1011] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:16 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:17.406417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:17.407028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:17.413478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:17.413965 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:17.414399 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:17.415089 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:17.415543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:17.415968 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:17.543094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:17.551208 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-07d4c5c0-8910-42a6-bb0f-ee7732e44487 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.148656 Oct 12 02:38:17.551767 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 507/1012] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:17 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:18.574088 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:18.574753 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:18.581163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:18.581629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:18.582057 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:18.582737 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:18.583498 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:18.583596 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:18.708897 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:18.716929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3db33316-28c3-4786-ad99-0851613f3530 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.146815 Oct 12 02:38:18.717491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 506/1013] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:18 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:19.741339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:19.741949 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:19.748339 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:19.748848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:19.749306 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:19.749993 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:19.750446 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:19.750868 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:19.879808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:19.888582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-517b171b-303c-4fc6-b35f-84d11e11335c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.151268 Oct 12 02:38:19.889180 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 508/1014] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:19 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:20.911421 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:20.912060 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:20.918684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:20.919167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:20.919598 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:20.920278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:20.920755 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:20.921177 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:21.051620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:21.059704 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-241eccf3-2ce2-4943-9047-0ff6ced2bb2c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1721 microversion: 2.1 time: 0.152135 Oct 12 02:38:21.060263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 507/1015] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:20 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:22.082562 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:22.083160 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:22.089546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:22.090024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:22.090957 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:22.091631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:22.092078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:22.092493 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:22.218743 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:22.226934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8fbebb7b-59a8-458b-9473-e1aa14dc6eb7 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1698 microversion: 2.1 time: 0.148288 Oct 12 02:38:22.227504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 509/1016] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:22 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1698 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:22.249454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ca463d71-770b-429b-b9c2-d192f244f159 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:38:22.255886 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca463d71-770b-429b-b9c2-d192f244f159 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:22.256363 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca463d71-770b-429b-b9c2-d192f244f159 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:22.256818 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ca463d71-770b-429b-b9c2-d192f244f159 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:22.317124 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ca463d71-770b-429b-b9c2-d192f244f159 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Going to try to start instance {{(pid=118865) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 12 02:38:22.358360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ca463d71-770b-429b-b9c2-d192f244f159 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action" status: 202 len: 0 microversion: 2.1 time: 0.112494 Oct 12 02:38:22.358876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 508/1017] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:38:22 2022] POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:38:22.366970 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:22.367584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:22.373587 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:22.374077 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:22.374543 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:22.375239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:22.375698 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:22.376123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:22.532684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:22.540945 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b8a4d44e-6e76-4350-b93c-90f05c4e1ecc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1707 microversion: 2.1 time: 0.177660 Oct 12 02:38:22.541389 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 510/1018] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:22 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1707 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:23.565346 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:23.566038 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:23.572042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:23.572533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:23.573000 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:23.573685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:23.574140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:23.575178 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:23.700723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:23.708725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f237ee65-dcb7-486a-9249-28ffcec20f69 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1707 microversion: 2.1 time: 0.148102 Oct 12 02:38:23.709299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 509/1019] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:23 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1707 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:24.731617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:24.732230 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:24.738754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:24.739240 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:24.739684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:24.740370 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:24.740855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:24.741281 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:24.867400 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:24.876144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6c6ad8b1-bf90-476e-9d38-14c5d85958ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1711 microversion: 2.1 time: 0.148768 Oct 12 02:38:24.876830 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 511/1020] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:24 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1711 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:42.385486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:38:42.386059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:42.388579 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=16 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:42.389267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:42.395278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:42.395759 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:42.396192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:42.396909 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:42.397360 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:42.397777 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:42.421931 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lazy-loading 'flavor' on Instance uuid 3d484497-9de8-43e6-b71c-d3a2d2488db8 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:38:42.490751 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4d6a4e0b-e056-4ed2-b012-50657f9b43e6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "POST /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8/action" status: 202 len: 0 microversion: 2.1 time: 0.109441 Oct 12 02:38:42.491330 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 510/1021] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:38:42 2022] POST /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8/action => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:38:42.499542 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:42.500156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:42.502754 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=16 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:42.503425 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:38:42.509634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:42.510125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:42.510564 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:42.511264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:42.511723 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:42.512149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:42.646057 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:42.654301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-68f4d5fc-c9ae-4360-9b39-eaedfc6970b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.158080 Oct 12 02:38:42.654899 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 512/1022] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:42 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:43.678714 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:43.679424 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:43.685513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:43.685987 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:43.686410 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:43.687078 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:43.687517 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:43.687925 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:43.818565 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:43.826262 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-47c1e3f2-00fa-4f60-adb6-f2c4975738b5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.152215 Oct 12 02:38:43.826821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 511/1023] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:43 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:38:44.850533 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:44.851058 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:44.858162 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:44.858646 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:44.859080 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:44.859765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:44.860220 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:44.860643 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:44.988130 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:44.995785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-745b2181-e2dd-4282-a9d6-5c7d0133a7fa tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.149190 Oct 12 02:38:44.996341 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 513/1024] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:44 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:46.019175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:46.019776 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:46.026091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:46.026564 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:46.026992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:46.027660 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:46.028103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:46.028536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:46.158820 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:46.166634 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3187b9cc-567b-4466-b181-81459e312df3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.151584 Oct 12 02:38:46.167185 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 512/1025] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:46 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:47.190582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:47.191123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:47.197482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:47.197962 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:47.198391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:47.199102 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:47.199554 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:47.199972 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:47.331999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:47.339812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-03f5fc70-722f-4791-8153-4c23a4285a9f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.153222 Oct 12 02:38:47.340384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 514/1026] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:47 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:48.363744 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:48.364351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:48.370805 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:48.371233 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:48.371716 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:48.372532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:48.373080 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:48.373538 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:48.507509 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:48.515650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-55adf0fc-1fa2-46fb-b0a9-0b8eebdde5a4 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.155900 Oct 12 02:38:48.516216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 513/1027] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:48 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:49.539129 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:49.539732 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:49.546437 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:49.546913 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:49.547345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:49.548018 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:49.548468 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:49.548918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:49.677810 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:49.685279 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3255446c-2015-418b-b974-58226fc95874 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.150022 Oct 12 02:38:49.685859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 515/1028] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:49 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:50.709107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:50.709644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:50.715968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:50.716449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:50.716908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:50.717666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:50.718609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:50.719025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:50.847059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:50.855778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e3f29821-78f6-4c57-a5cb-8d6b36d392e3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.150583 Oct 12 02:38:50.856351 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 514/1029] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:50 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:51.880524 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:51.881151 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:51.887732 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:51.888242 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:51.888674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:51.889381 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:51.889837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:51.890264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:52.020921 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:52.028814 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-611585bf-abf8-4cf0-8684-180d84fa7397 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.152108 Oct 12 02:38:52.029380 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 516/1030] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:51 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:53.052246 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:53.052940 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:53.059358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:53.059840 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:53.060277 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:53.060998 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:53.061482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:53.061938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:53.190981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:53.198612 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d5f1f915-1c81-435d-93ce-935208e6d2cf tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.150315 Oct 12 02:38:53.199167 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 515/1031] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:53 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:54.222651 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:54.223258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:54.229950 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:54.230429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:54.230860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:54.231538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:54.231988 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:54.232406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:54.362414 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:54.370166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2dab8802-5b85-4e9b-a783-e9851946f9c6 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.151318 Oct 12 02:38:54.370668 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 517/1032] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:54 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:55.393247 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:55.393960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:55.400449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:55.401040 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:55.401481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:55.402168 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:55.402623 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:55.403044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:55.538639 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:55.547393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9cbdd90e-39b7-473b-b76d-a8b0ae6ecd56 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1568 microversion: 2.1 time: 0.157990 Oct 12 02:38:55.547951 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 516/1033] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:55 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:55.569762 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:38:55.570358 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:55.576660 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:55.577173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:55.577604 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:55.578282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:55.578736 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:55.579154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:55.620441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118864) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 12 02:38:55.683876 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-81e5cc97-6b13-412e-a2c9-4497137504f5 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "POST /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8/action" status: 202 len: 0 microversion: 2.1 time: 0.117970 Oct 12 02:38:55.684483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 518/1034] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:38:55 2022] POST /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8/action => generated 0 bytes in 119 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:38:55.692791 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:55.693477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:55.699490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:55.699952 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:55.700376 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:55.701113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:55.701561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:55.701971 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:55.848825 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:55.857066 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4924c315-4887-4d07-a251-09c6b72c77fc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1576 microversion: 2.1 time: 0.168029 Oct 12 02:38:55.857635 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 517/1035] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:55 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1576 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:56.881192 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:56.881721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:56.887603 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:56.888078 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:56.888501 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:56.889196 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:56.889645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:56.890059 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:57.023091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:57.031264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3970707a-bb12-44bd-8721-14d3689d1c1f tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1516 microversion: 2.1 time: 0.154483 Oct 12 02:38:57.031824 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 519/1036] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:56 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1516 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:57.817604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:57.818184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:57.824444 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:57.824974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:57.825399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:57.826071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:57.827170 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:57.827587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:57.905614 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.objects.instance [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lazy-loading 'services' on Instance uuid f5970385-f587-4e45-b2ca-9d5913f90396 {{(pid=118865) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:38:57.974144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4e04fc4b-7a47-4f19-9707-4a549d93ffcc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-volume_attachments/c3c69415-89ce-40eb-95d5-7db7e1759f03" status: 202 len: 0 microversion: 2.1 time: 0.160272 Oct 12 02:38:57.974727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 518/1037] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:38:57 2022] DELETE /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-volume_attachments/c3c69415-89ce-40eb-95d5-7db7e1759f03 => generated 0 bytes in 162 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 12 02:38:58.054753 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:58.055379 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:58.062377 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:58.062864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:58.063303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:58.064000 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:58.064458 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:58.064911 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:58.192545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:58.200332 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9c2c3b48-b3bf-4e8c-b93a-81201ee8548d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1516 microversion: 2.1 time: 0.149482 Oct 12 02:38:58.200931 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 520/1038] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:58 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1516 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:38:58.418345 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-29fdb73d-812c-4355-bdaf-81df9692db96 req-3dec8899-fcf6-4c29-8dde-be35dda59233 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d484497-9de8-43e6-b71c-d3a2d2488db8", "name": "network-vif-unplugged", "status": "completed", "tag": "66a8b57d-c6be-4a74-91b2-545572f47798"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:38:58.441327 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-29fdb73d-812c-4355-bdaf-81df9692db96 req-3dec8899-fcf6-4c29-8dde-be35dda59233 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:58.441810 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-29fdb73d-812c-4355-bdaf-81df9692db96 req-3dec8899-fcf6-4c29-8dde-be35dda59233 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:58.442238 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-29fdb73d-812c-4355-bdaf-81df9692db96 req-3dec8899-fcf6-4c29-8dde-be35dda59233 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:58.453379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.compute.server_external_events [req-29fdb73d-812c-4355-bdaf-81df9692db96 req-3dec8899-fcf6-4c29-8dde-be35dda59233 service nova] Unable to find a host for instance 3d484497-9de8-43e6-b71c-d3a2d2488db8. Dropping event network-vif-unplugged {{(pid=118865) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 12 02:38:58.454254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-29fdb73d-812c-4355-bdaf-81df9692db96 req-3dec8899-fcf6-4c29-8dde-be35dda59233 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.040346 Oct 12 02:38:58.455023 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 519/1039] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:38:58 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 42 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 12 02:38:59.093133 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fbdab97c-de73-4fee-ad91-9e6688646a53 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:38:59.099319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fbdab97c-de73-4fee-ad91-9e6688646a53 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:59.100023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fbdab97c-de73-4fee-ad91-9e6688646a53 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:59.100453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fbdab97c-de73-4fee-ad91-9e6688646a53 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:59.125851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fbdab97c-de73-4fee-ad91-9e6688646a53 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Going to try to stop instance {{(pid=118864) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 12 02:38:59.170886 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fbdab97c-de73-4fee-ad91-9e6688646a53 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action" status: 202 len: 0 microversion: 2.1 time: 0.082081 Oct 12 02:38:59.171673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 521/1040] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:38:59 2022] POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action => generated 0 bytes in 83 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:38:59.179602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:59.180465 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:59.186763 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:59.187518 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:59.187900 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:59.188801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:59.189282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:59.189702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:59.224322 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:38:59.225070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:38:59.231361 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:59.231847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:59.232285 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:59.232999 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:38:59.233452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:38:59.233869 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:38:59.327331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:59.335259 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ff7ff073-8a01-4f16-b5d1-69e7fadc9744 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.159489 Oct 12 02:38:59.335992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 520/1041] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:59 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:38:59.368864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:38:59.377039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-2aa3f87e-b3b4-497b-8090-16e1b638248b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1516 microversion: 2.1 time: 0.156883 Oct 12 02:38:59.377602 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 522/1042] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:38:59 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1516 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:00.359217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:00.359827 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:00.366349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:00.366754 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:00.367189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:00.367868 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:00.368322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:00.368772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:00.401199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:00.401811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:00.408853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:00.409335 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:00.409764 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:00.410484 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:00.410932 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:00.411347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:00.497025 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:00.504903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c4d029c9-b15a-4e62-8942-d9683e995e33 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.150066 Oct 12 02:39:00.505509 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 521/1043] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:00 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:00.543710 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:00.552567 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6afa6e5f-11f5-468a-82b6-46663a10ef34 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1516 microversion: 2.1 time: 0.155084 Oct 12 02:39:00.553318 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 523/1044] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:00 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1516 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:01.528314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:01.528945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:01.535928 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:01.536409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:01.536883 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:01.537612 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:01.538067 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:01.538494 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:01.576327 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:01.577128 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:01.583597 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:01.584087 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:01.584523 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:01.585245 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:01.585701 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:01.586123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:01.667306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:01.675913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3e6b9c1a-6386-4911-9b90-c944fc822d52 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.151370 Oct 12 02:39:01.676491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 522/1045] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:01 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:01.719853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:01.727779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a6605717-8151-43e1-8488-6c9d702c5721 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.155225 Oct 12 02:39:01.728340 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 524/1046] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:01 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:02.698874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:02.699602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:02.705997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:02.706477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:02.706941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:02.707617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:02.708069 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:02.708487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:02.752155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:02.752787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:02.759156 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:02.759643 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:02.760086 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:02.760805 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:02.761262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:02.761691 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:02.841992 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:02.849705 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6f9215bd-dce6-4b5d-9bec-c196b8991cb9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.154643 Oct 12 02:39:02.850277 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 523/1047] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:02 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:02.894939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:02.902448 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-25334f2b-6dde-4c7c-8cb6-6c08e5f42202 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.154830 Oct 12 02:39:02.903006 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 525/1048] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:02 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:03.873865 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:03.874461 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:03.881006 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:03.881480 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:03.881910 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:03.882607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:03.883056 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:03.883470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:03.901343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:39:03.925024 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:03.925655 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:03.932177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:03.932667 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:03.933135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:03.933828 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:03.934282 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:03.934703 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:03.953941 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:39:04.014550 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:04.023083 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-15b806bb-655c-4f8b-a555-c9166d96eb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.153549 Oct 12 02:39:04.023894 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 524/1049] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:03 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:04.068819 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:04.076371 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9fe3be47-478f-4a8f-af5f-058cbb3f4bdc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.155317 Oct 12 02:39:04.076955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 526/1050] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:03 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:05.045033 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:05.045647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:05.052438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:05.052956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:05.053394 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:05.054076 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:05.054533 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:05.054955 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:05.099239 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:05.099903 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:05.106769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:05.107326 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:05.107765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:05.108951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:05.109419 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:05.109836 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:05.188307 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:05.195791 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-678914d0-6155-4af2-a581-94539b961a91 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.154551 Oct 12 02:39:05.196365 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 525/1051] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:05 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:05.240305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:05.248023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-aacc0ba1-17d8-4839-b29e-eb5190953cc2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.152618 Oct 12 02:39:05.248581 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 527/1052] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:05 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:06.220572 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:06.221248 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:06.227859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:06.228341 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:06.228812 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:06.229505 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:06.229958 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:06.230378 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:06.271441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:06.272182 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:06.278801 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:06.279295 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:06.279731 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:06.280483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:06.280964 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:06.281391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:06.359746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:06.367552 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-9ac8ba33-3f70-4136-9ce7-baf91ac37b60 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.150752 Oct 12 02:39:06.368059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 526/1053] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:06 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:06.411329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:06.418983 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c6df0942-b9b9-4387-8aed-35e681a53acc tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.151301 Oct 12 02:39:06.419549 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 528/1054] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:06 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:07.390911 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:07.391423 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:07.397876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:07.398358 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:07.398793 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:07.399503 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:07.399957 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:07.400406 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:07.443205 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:07.443852 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:07.450794 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:07.451287 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:07.451718 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:07.453105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:07.453559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:07.453975 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:07.590812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:07.598576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5024c343-e1aa-4791-a461-b8d9e01d784c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.159263 Oct 12 02:39:07.599141 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 529/1055] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:07 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:07.676876 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:07.684399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-afca0480-9a1d-4514-90f5-db5ebb5dbd67 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.297904 Oct 12 02:39:07.684985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 527/1056] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:07 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:08.621721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:08.622335 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:08.628614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:08.629125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:08.629582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:08.630264 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:08.630717 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:08.631135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:08.707837 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:08.708500 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:08.714802 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:08.715285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:08.715752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:08.716434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:08.716921 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:08.717450 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:08.763576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:08.771916 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c6832f13-4f9f-4da3-94bd-2d9f701825da tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.154002 Oct 12 02:39:08.772490 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 530/1057] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:08 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:08.847930 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:08.855794 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-93fbb148-c0d8-4605-a061-7e2b0be53b2b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1675 microversion: 2.1 time: 0.151737 Oct 12 02:39:08.856544 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 528/1058] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:08 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1675 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:09.795417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:09.796515 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:09.802847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:09.803324 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:09.803758 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:09.804444 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:09.804927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:09.805347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:09.878685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:09.879264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:09.885528 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:09.886007 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:09.886437 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:09.887123 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:09.887567 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:09.887981 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:09.934379 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:09.942123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0175034f-7065-453e-a1b0-a5da3f071117 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.150659 Oct 12 02:39:09.942689 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 531/1059] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:09 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:10.014728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:10.022352 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e7f87f3b-c35d-48cc-bde6-035c377b80f0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1652 microversion: 2.1 time: 0.147468 Oct 12 02:39:10.022899 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 529/1060] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:09 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1652 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:10.044873 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3404a611-974e-4952-acb4-b80403876ba9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:39:10.051403 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3404a611-974e-4952-acb4-b80403876ba9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:10.051829 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3404a611-974e-4952-acb4-b80403876ba9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:10.052262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3404a611-974e-4952-acb4-b80403876ba9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:10.073417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3404a611-974e-4952-acb4-b80403876ba9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Going to try to start instance {{(pid=118864) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 12 02:39:10.115107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3404a611-974e-4952-acb4-b80403876ba9 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action" status: 202 len: 0 microversion: 2.1 time: 0.073898 Oct 12 02:39:10.115701 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 532/1061] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 12 02:39:10 2022] POST /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/action => generated 0 bytes in 75 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:39:10.124044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:10.124769 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:10.131263 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:10.131869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:10.132307 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:10.133153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:10.134117 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:10.134542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:10.268364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:10.276158 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4cb38e6b-0d73-451a-b5b3-f932f8b0f6a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1661 microversion: 2.1 time: 0.155853 Oct 12 02:39:10.276739 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 530/1062] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:10 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1661 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:10.540769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-5ff5630f-839a-4fa6-aac9-5650d686c697 req-6543b183-ea13-4878-a1f4-ef9604ae28a7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d484497-9de8-43e6-b71c-d3a2d2488db8", "name": "network-vif-plugged", "status": "completed", "tag": "66a8b57d-c6be-4a74-91b2-545572f47798"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:39:10.551221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5ff5630f-839a-4fa6-aac9-5650d686c697 req-6543b183-ea13-4878-a1f4-ef9604ae28a7 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:10.551765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5ff5630f-839a-4fa6-aac9-5650d686c697 req-6543b183-ea13-4878-a1f4-ef9604ae28a7 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:10.552241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-5ff5630f-839a-4fa6-aac9-5650d686c697 req-6543b183-ea13-4878-a1f4-ef9604ae28a7 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:10.569054 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-5ff5630f-839a-4fa6-aac9-5650d686c697 req-6543b183-ea13-4878-a1f4-ef9604ae28a7 service nova] Creating event network-vif-plugged:66a8b57d-c6be-4a74-91b2-545572f47798 for instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 on hv-cinder-84914 Oct 12 02:39:10.574595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-5ff5630f-839a-4fa6-aac9-5650d686c697 req-6543b183-ea13-4878-a1f4-ef9604ae28a7 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037929 Oct 12 02:39:10.575349 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 533/1063] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:39:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:39:10.965180 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:10.965923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:10.972570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:10.973093 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:10.973530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:10.974223 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:10.974679 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:10.975100 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:11.102608 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:11.110801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1652fc9f-3be9-4f77-ae70-478290683bc3 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.149476 Oct 12 02:39:11.111387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 531/1064] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:10 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:11.302246 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:11.303298 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:11.310165 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:11.310664 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:11.311081 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:11.311770 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:11.312225 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:11.312645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:11.444287 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:11.452161 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e6977c73-3384-4217-99a4-439fbc3a1f11 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1661 microversion: 2.1 time: 0.155106 Oct 12 02:39:11.452756 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 534/1065] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:11 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1661 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:12.134434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:12.135044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:12.142056 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:12.142535 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:12.142965 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:12.143653 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:12.144123 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:12.144546 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:12.283673 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:12.291682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a9732c8c-8faa-4d1a-9929-97c29106e40b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1574 microversion: 2.1 time: 0.161093 Oct 12 02:39:12.292240 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 532/1066] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:12 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1574 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:12.475671 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:12.476280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:12.482939 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:12.483920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:12.484354 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:12.485071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:12.485525 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:12.485944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:12.613273 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:12.621057 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-55f3b632-a4d3-4588-b6d5-01c4c52e82ba tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1665 microversion: 2.1 time: 0.149226 Oct 12 02:39:12.621619 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 535/1067] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:12 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1665 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:13.314686 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:13.315290 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:13.317557 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=47 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:39:13.322687 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:13.323163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:13.323589 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:13.324264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:13.324727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:13.325143 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:13.454584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:13.462227 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-1e7976bc-95a4-4948-a01c-85ba8d5b8ec8 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1561 microversion: 2.1 time: 0.151371 Oct 12 02:39:13.462797 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 533/1068] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:13 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:45.270206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:45.272451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:45.274721 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:39:45.275383 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:39:45.276631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:39:45.276809 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Wed Oct 12 02:39:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-volume_attachments (ip 10.222.0.23) !!! Oct 12 02:39:45.285739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:45.286147 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:45.286584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:45.287303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:45.287756 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:45.288175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:45.319385 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-376e634a-75f4-4a56-ad61-618f5167b0a6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053184 Oct 12 02:39:45.319940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 536/1069] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:39:45 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:39:45.382418 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:45.382908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:45.384923 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:39:45.385585 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=56 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:39:45.391597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:45.392077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:45.392508 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:45.393213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:45.393667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:45.394084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:45.419215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:39:45.493874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3fa9a1e6-ba68-4f41-bf61-1aac73e9df34 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 204 len: 0 microversion: 2.1 time: 0.115043 Oct 12 02:39:45.494548 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 534/1070] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:39:45 2022] DELETE /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:39:45.502927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:45.503552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:45.509633 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:45.510117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:45.510548 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:45.511231 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:45.511751 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:45.512175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:45.686521 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:45.694274 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-063ad2aa-86b9-48d2-8da7-fa9b27fcf628 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1671 microversion: 2.1 time: 0.195258 Oct 12 02:39:45.694904 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 537/1071] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:45 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1671 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:46.717762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:46.718458 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:46.724499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:46.725015 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:46.725707 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:46.726385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:46.726833 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:46.727277 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:46.906442 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:46.914561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-8d7dfd8d-c34f-48e5-a639-e6a9de3608a8 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1671 microversion: 2.1 time: 0.200993 Oct 12 02:39:46.915116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 535/1072] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:46 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1671 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:47.939211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:47.939853 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:47.946307 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:47.946787 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:47.947216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:47.947912 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:47.948365 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:47.948831 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:48.166649 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:48.175391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ed2f4a05-418d-469c-a511-fcbcbf71ab1b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 200 len: 1671 microversion: 2.1 time: 0.240676 Oct 12 02:39:48.175967 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 538/1073] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:47 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 1671 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:48.603097 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-f8140e46-bfbc-4bd2-aa9e-873a0e43d9e4 req-952dc6b7-849d-440c-afea-0279dc38e45e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5970385-f587-4e45-b2ca-9d5913f90396", "name": "network-vif-deleted", "tag": "24ac17dd-767d-4f70-9eec-365ba24671e0"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:39:48.615872 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f8140e46-bfbc-4bd2-aa9e-873a0e43d9e4 req-952dc6b7-849d-440c-afea-0279dc38e45e service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:48.616370 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f8140e46-bfbc-4bd2-aa9e-873a0e43d9e4 req-952dc6b7-849d-440c-afea-0279dc38e45e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:48.617052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-f8140e46-bfbc-4bd2-aa9e-873a0e43d9e4 req-952dc6b7-849d-440c-afea-0279dc38e45e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:48.634052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-f8140e46-bfbc-4bd2-aa9e-873a0e43d9e4 req-952dc6b7-849d-440c-afea-0279dc38e45e service nova] Creating event network-vif-deleted:24ac17dd-767d-4f70-9eec-365ba24671e0 for instance f5970385-f587-4e45-b2ca-9d5913f90396 on hv-cinder-84914 Oct 12 02:39:48.642047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-f8140e46-bfbc-4bd2-aa9e-873a0e43d9e4 req-952dc6b7-849d-440c-afea-0279dc38e45e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041985 Oct 12 02:39:48.643602 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 536/1074] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:39:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:39:49.201394 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:49.203150 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:49.211951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:49.212564 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:49.213084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:49.215044 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:49.215630 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:49.216148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:49.232499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] HTTP exception thrown: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Request method failure captured: Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 HTTP/1.1 Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRihb5UIAeD_NP7kC_ZFsPo1l7SWXoUv7g_UGBxRAHC7_uJJaiN2yn8aP0n-5xK3fEnWR67d5TD0kTMIK-k3kM64hiafEzYEyLTBsbpN7XKlozvmm67R__jnUzX1c2w6EVsnn3phUUBw0Ddk02eXi21nqFeep6_L6zUUZHb5WYXF6xHw Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:39:49.234207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_297b4af1911c49f3b7b01c35236f4dae", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: ea7aceaca15a47f89c674c237b0cacae Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:39:49.236451 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'f5970385-f587-4e45-b2ca-9d5913f90396'} Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:39:49.237481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:39:49.238951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:39:49.240323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Returning 404 to user: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:39:49.240323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d24674ee-bd3d-4807-af61-ff28c167efdf tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 404 len: 111 microversion: 2.1 time: 0.040272 Oct 12 02:39:49.240323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 539/1075] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:49 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:39:49.245611 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a06b35ee-bdc1-41c7-8775-69e0a793cefb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '.version_select at 0x7f59d34551f0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:49.265570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a06b35ee-bdc1-41c7-8775-69e0a793cefb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-914788794" status: 202 len: 0 microversion: 2.1 time: 0.024002 Oct 12 02:39:49.266331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 537/1076] 10.222.0.23 () {58 vars in 1218 bytes} [Wed Oct 12 02:39:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-914788794 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:39:49.895356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8dd7d32f-30ae-4181-9a05-0f534f88fd1a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'create', calling method: .version_select at 0x7f59d3693790>, body: {"keypair": {"name": "tempest-keypair-1240949202", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLTcCtndWjSRI81Hnn0+IYCWZxKprz7NvszAr2ZaQl45pv2i+yzM7EteBCFEATtgp3fDhBkHIk9ozJKTJDWM5/1xhDypbLc7K59ETx/85+FiExHI1PsnQjzyimnoaAkSUw=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:39:49.910765 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-8dd7d32f-30ae-4181-9a05-0f534f88fd1a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:39:49.914536 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-8dd7d32f-30ae-4181-9a05-0f534f88fd1a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for user ea7aceaca15a47f89c674c237b0cacae and project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:39:49.939211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8dd7d32f-30ae-4181-9a05-0f534f88fd1a tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.048082 Oct 12 02:39:49.939799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 540/1077] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:39:49 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:39:51.429997 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1465271275", "imageRef": "0315a9a6-a11d-4efd-b250-80b576060fc3", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8484fb9c-5daa-4067-a233-ae010b930f21"}], "security_groups": [{"name": "tempest-securitygroup--227333509"}], "key_name": "tempest-keypair-1240949202", "user_data": "}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:39:51.530859 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] validate_networks() for [('8484fb9c-5daa-4067-a233-ae010b930f21', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:39:51.942067 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:39:51.942519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:39:51.943065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:39:51.943476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:39:51.943888 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:39:51.952746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:51.953188 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:51.954118 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:51.954840 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:51.955296 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:51.955718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:51.973033 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:39:51.977194 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Getting quotas for user ea7aceaca15a47f89c674c237b0cacae and project 297b4af1911c49f3b7b01c35236f4dae. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:39:51.988198 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:39:51.990578 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='0315a9a6-a11d-4efd-b250-80b576060fc3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:39:52.037148 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e0561bda-451a-44b0-92bb-26401948495c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.611341 Oct 12 02:39:52.037969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 538/1078] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:39:51 2022] POST /compute/v2.1/servers => generated 401 bytes in 613 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:39:52.050323 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-503c37b2-d68d-4550-b189-35fa10074b65 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:52.051008 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-503c37b2-d68d-4550-b189-35fa10074b65 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:52.108540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-503c37b2-d68d-4550-b189-35fa10074b65 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:52.116335 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-503c37b2-d68d-4550-b189-35fa10074b65 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1238 microversion: 2.1 time: 0.070914 Oct 12 02:39:52.117112 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 541/1079] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:52 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1238 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:53.140068 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:53.141877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:53.148004 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:53.148692 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:53.149390 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:53.150081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:53.150735 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:53.151349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:53.215104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:53.223945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ce29986e-1a88-48ba-b4fb-aa285f2cdb41 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1308 microversion: 2.1 time: 0.088228 Oct 12 02:39:53.224727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 539/1080] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:53 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1308 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:54.247803 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:54.248657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:54.255190 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:54.255701 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:54.256135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:54.256902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:54.257359 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:54.257780 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:54.473611 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:54.481622 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-26457421-9ac3-4302-b5c5-322ce7034667 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1363 microversion: 2.1 time: 0.237978 Oct 12 02:39:54.482127 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 542/1081] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:54 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1363 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:54.620102 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-d56aea13-17b8-4cc8-a0f7-3c38ed54dc6b req-7343e0df-af46-4e84-9d58-58fcfa6035b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50e73cb0-a161-4344-be58-ebf9dc84c4a2", "tag": "e1dfc227-256b-42f2-8bbc-5b489c6f72d3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:39:54.630885 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-d56aea13-17b8-4cc8-a0f7-3c38ed54dc6b req-7343e0df-af46-4e84-9d58-58fcfa6035b7 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:54.631478 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-d56aea13-17b8-4cc8-a0f7-3c38ed54dc6b req-7343e0df-af46-4e84-9d58-58fcfa6035b7 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:54.631979 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-d56aea13-17b8-4cc8-a0f7-3c38ed54dc6b req-7343e0df-af46-4e84-9d58-58fcfa6035b7 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:54.646781 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-d56aea13-17b8-4cc8-a0f7-3c38ed54dc6b req-7343e0df-af46-4e84-9d58-58fcfa6035b7 service nova] Creating event network-changed:e1dfc227-256b-42f2-8bbc-5b489c6f72d3 for instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 on hv-cinder-84914 Oct 12 02:39:54.651752 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-d56aea13-17b8-4cc8-a0f7-3c38ed54dc6b req-7343e0df-af46-4e84-9d58-58fcfa6035b7 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035351 Oct 12 02:39:54.652352 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 540/1082] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:39:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:39:55.505563 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:55.506290 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:55.512716 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:55.513213 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:55.513644 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:55.514329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:55.514782 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:55.515204 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:55.641441 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:55.649559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8b777190-1748-4ebc-a2ad-44f4583322d0 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1363 microversion: 2.1 time: 0.147819 Oct 12 02:39:55.650126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 543/1083] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:55 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1363 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:56.674666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:56.675410 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:56.681408 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:56.681879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:56.682306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:56.682977 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:56.683422 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:56.683834 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:56.815482 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:56.822941 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-98c9f8d8-e430-4d5a-b4cb-42a5da55ae19 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1363 microversion: 2.1 time: 0.153629 Oct 12 02:39:56.823496 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 541/1084] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:56 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1363 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:39:57.845426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:57.846614 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:57.853201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:57.853687 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:57.854120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:57.854804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:57.855256 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:57.855676 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:57.986000 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:57.993478 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6fb452f3-59c4-45ed-a12b-d9b96fad49fc tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1363 microversion: 2.1 time: 0.151943 Oct 12 02:39:57.994042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 544/1085] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:57 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1363 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:39:59.016667 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:39:59.017291 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:39:59.023796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:59.024268 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:59.024723 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:59.025393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:39:59.025839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:39:59.026254 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:39:59.157819 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:39:59.166256 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4d04e445-ca54-429c-ba7d-6e61763892fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1363 microversion: 2.1 time: 0.153425 Oct 12 02:39:59.166821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 542/1086] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:39:59 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1363 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:40:00.188293 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:00.188928 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:00.195283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:00.195766 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:00.196202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:00.196926 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:00.197384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:00.197804 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:00.323094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:40:00.330753 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-1adf8f89-925d-41f3-bf9a-8e6b3e271728 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1363 microversion: 2.1 time: 0.146297 Oct 12 02:40:00.331368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 545/1087] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:40:00 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1363 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:40:00.754323 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-034860fa-1285-424a-a279-e62f781c8901 req-a55175ee-c4c1-4a08-8585-25c4d79c8a30 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "50e73cb0-a161-4344-be58-ebf9dc84c4a2", "name": "network-vif-plugged", "status": "completed", "tag": "e1dfc227-256b-42f2-8bbc-5b489c6f72d3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:40:00.764122 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-034860fa-1285-424a-a279-e62f781c8901 req-a55175ee-c4c1-4a08-8585-25c4d79c8a30 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:00.764605 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-034860fa-1285-424a-a279-e62f781c8901 req-a55175ee-c4c1-4a08-8585-25c4d79c8a30 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:00.765062 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-034860fa-1285-424a-a279-e62f781c8901 req-a55175ee-c4c1-4a08-8585-25c4d79c8a30 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:00.776395 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-034860fa-1285-424a-a279-e62f781c8901 req-a55175ee-c4c1-4a08-8585-25c4d79c8a30 service nova] Creating event network-vif-plugged:e1dfc227-256b-42f2-8bbc-5b489c6f72d3 for instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 on hv-cinder-84914 Oct 12 02:40:00.781303 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-034860fa-1285-424a-a279-e62f781c8901 req-a55175ee-c4c1-4a08-8585-25c4d79c8a30 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030640 Oct 12 02:40:00.781879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 543/1088] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:40:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:40:01.353637 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:01.354173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:01.360612 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:01.361123 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:01.361555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:01.362238 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:01.363206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:01.363628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:01.491901 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:40:01.499626 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e8c2ea1c-79a5-4692-83c0-d3c803ef99ab tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1363 microversion: 2.1 time: 0.150126 Oct 12 02:40:01.500200 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 546/1089] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:40:01 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1363 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:40:02.521567 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:02.522451 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=6 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:02.526950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:02.527650 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:02.535473 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:02.535975 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:02.536398 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:02.537212 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:02.537671 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:02.538098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:02.667486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:40:02.675153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-423a75de-e165-4af8-af97-c58c01bef175 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1549 microversion: 2.1 time: 0.155300 Oct 12 02:40:02.675747 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 544/1090] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:40:02 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1549 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:40:02.697974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:02.698491 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:02.700257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=6 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:02.706725 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:02.707203 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:02.707637 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:02.708311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:02.708782 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:02.709199 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:02.785029 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5972bda9-87f6-4681-a419-861bd5781454 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090874 Oct 12 02:40:02.785631 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 547/1091] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 12 02:40:02 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:03.680526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-479b4f15-da72-4e60-831d-ebd582399f11 req-e4f97c3e-1c20-4d23-9962-59af7c4b47a0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50e73cb0-a161-4344-be58-ebf9dc84c4a2", "tag": "e1dfc227-256b-42f2-8bbc-5b489c6f72d3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:40:03.690536 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-479b4f15-da72-4e60-831d-ebd582399f11 req-e4f97c3e-1c20-4d23-9962-59af7c4b47a0 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:03.691022 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-479b4f15-da72-4e60-831d-ebd582399f11 req-e4f97c3e-1c20-4d23-9962-59af7c4b47a0 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:03.691452 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-479b4f15-da72-4e60-831d-ebd582399f11 req-e4f97c3e-1c20-4d23-9962-59af7c4b47a0 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:03.703206 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-479b4f15-da72-4e60-831d-ebd582399f11 req-e4f97c3e-1c20-4d23-9962-59af7c4b47a0 service nova] Creating event network-changed:e1dfc227-256b-42f2-8bbc-5b489c6f72d3 for instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 on hv-cinder-84914 Oct 12 02:40:03.708864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-479b4f15-da72-4e60-831d-ebd582399f11 req-e4f97c3e-1c20-4d23-9962-59af7c4b47a0 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031848 Oct 12 02:40:03.709658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 545/1092] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:40:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:28.233056 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:28.233871 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=54 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:28.239719 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:28.240772 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:28.247621 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:28.248125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:28.257951 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:28.258422 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:28.258848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:28.261296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:28.262811 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:28.265703 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:28.293316 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:40:28.309945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-033e51e6-c9a0-44e6-ad8b-5e8185938fb5 req-633284f0-c7b3-43b0-b6e2-a837f567d0de service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3d484497-9de8-43e6-b71c-d3a2d2488db8", "tag": "66a8b57d-c6be-4a74-91b2-545572f47798"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:40:28.319604 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-033e51e6-c9a0-44e6-ad8b-5e8185938fb5 req-633284f0-c7b3-43b0-b6e2-a837f567d0de service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:28.320306 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-033e51e6-c9a0-44e6-ad8b-5e8185938fb5 req-633284f0-c7b3-43b0-b6e2-a837f567d0de service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:28.320858 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-033e51e6-c9a0-44e6-ad8b-5e8185938fb5 req-633284f0-c7b3-43b0-b6e2-a837f567d0de service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:28.332147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-033e51e6-c9a0-44e6-ad8b-5e8185938fb5 req-633284f0-c7b3-43b0-b6e2-a837f567d0de service nova] Creating event network-changed:66a8b57d-c6be-4a74-91b2-545572f47798 for instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 on hv-cinder-84914 Oct 12 02:40:28.339131 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-033e51e6-c9a0-44e6-ad8b-5e8185938fb5 req-633284f0-c7b3-43b0-b6e2-a837f567d0de service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100687 Oct 12 02:40:28.340453 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 546/1093] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:40:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:28.379452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d887f403-7588-4e20-a579-c424b7156b8d tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 204 len: 0 microversion: 2.1 time: 0.147614 Oct 12 02:40:28.383495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 548/1094] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:40:28 2022] DELETE /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:40:28.391158 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:28.393097 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:28.401053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:28.401595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:28.402089 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:28.403422 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:28.404071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:28.405713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:28.544674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:40:28.552609 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-36ca7a3a-0a14-46a2-a5fa-de8f7394db31 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.166466 Oct 12 02:40:28.553278 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 547/1095] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:40:28 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:40:29.576402 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:29.577137 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:29.584091 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:29.584579 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:29.585040 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:29.585736 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:29.586193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:29.586617 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:29.744215 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:40:29.751953 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-26338b6b-2602-4ed6-9817-f89f4f81964c tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1567 microversion: 2.1 time: 0.179814 Oct 12 02:40:29.752517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 549/1096] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:40:29 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1567 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:40:30.646086 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-9ce4d2d0-7a0e-4d3b-af20-1235b782ea71 req-e3946077-2f6a-45fa-9f33-26759811ad16 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3d484497-9de8-43e6-b71c-d3a2d2488db8", "name": "network-vif-deleted", "tag": "66a8b57d-c6be-4a74-91b2-545572f47798"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:40:30.663385 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-9ce4d2d0-7a0e-4d3b-af20-1235b782ea71 req-e3946077-2f6a-45fa-9f33-26759811ad16 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:30.663870 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-9ce4d2d0-7a0e-4d3b-af20-1235b782ea71 req-e3946077-2f6a-45fa-9f33-26759811ad16 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:30.664300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-9ce4d2d0-7a0e-4d3b-af20-1235b782ea71 req-e3946077-2f6a-45fa-9f33-26759811ad16 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:30.679680 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-9ce4d2d0-7a0e-4d3b-af20-1235b782ea71 req-e3946077-2f6a-45fa-9f33-26759811ad16 service nova] Creating event network-vif-deleted:66a8b57d-c6be-4a74-91b2-545572f47798 for instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 on hv-cinder-84914 Oct 12 02:40:30.684422 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-9ce4d2d0-7a0e-4d3b-af20-1235b782ea71 req-e3946077-2f6a-45fa-9f33-26759811ad16 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042693 Oct 12 02:40:30.685039 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 548/1097] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:40:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:30.776206 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:30.776840 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:30.783181 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:30.783663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:30.784096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:30.784808 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:30.785267 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:30.786363 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:30.853023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '60e6b5fcfd9d454bb10dd314bcdbedf6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b1601a3e4f41405d8e9001af6da86b46', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:40:30.860758 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fff5ae05-23d7-4921-ae92-31c8276005d2 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 200 len: 1219 microversion: 2.1 time: 0.088368 Oct 12 02:40:30.861280 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 550/1098] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:40:30 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:40:31.884146 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:31.884854 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] [instance: 3d484497-9de8-43e6-b71c-d3a2d2488db8] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:31.891420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:31.891903 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:31.892335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:31.893057 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:31.893516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:31.893945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:31.906864 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] HTTP exception thrown: Instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 could not be found. Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Request method failure captured: Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 HTTP/1.1 Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRigHCqZFvWg6Dksip4Q3X5HmIQd3ePGx8NoEAW3cqhZwauYW14-qR3Uw6nvlMcUUqZQ8Cp_RBayPORpObDxtaLAYj1BHMDJcW_okdGP_bw80b0C9KFZj0H7DcZzNDP3fk7a8RzY6Ged5CRYsYSZMuS-rH8JCUg_7KzNJ5gz80jfSE0g Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: b1601a3e4f41405d8e9001af6da86b46 Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-TestShelveInstance-677938886 Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:40:31.908443 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/b1601a3e4f41405d8e9001af6da86b46"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/b1601a3e4f41405d8e9001af6da86b46"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/b1601a3e4f41405d8e9001af6da86b46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_b1601a3e4f41405d8e9001af6da86b46", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-TestShelveInstance-677938886 Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: b1601a3e4f41405d8e9001af6da86b46 Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-TestShelveInstance-677938886 Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-TestShelveInstance-677938886-project Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: 60e6b5fcfd9d454bb10dd314bcdbedf6 Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-TestShelveInstance-677938886-project Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 could not be found. Oct 12 02:40:31.910135 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '3d484497-9de8-43e6-b71c-d3a2d2488db8'} Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 could not be found. Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:40:31.911778 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 could not be found. Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 could not be found. Oct 12 02:40:31.913373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:40:31.915003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Returning 404 to user: Instance 3d484497-9de8-43e6-b71c-d3a2d2488db8 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:40:31.915003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-efdd14d0-2439-4fc9-938d-60b05a298b51 tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8" status: 404 len: 111 microversion: 2.1 time: 0.030097 Oct 12 02:40:31.915003 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 549/1099] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:40:31 2022] GET /compute/v2.1/servers/3d484497-9de8-43e6-b71c-d3a2d2488db8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:40:32.065272 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-6ee9914e-073d-4678-a814-02e129356e3b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] Calling method '.version_select at 0x7f59d3672e50>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:32.084667 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-6ee9914e-073d-4678-a814-02e129356e3b tempest-TestShelveInstance-677938886 tempest-TestShelveInstance-677938886-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-474987065" status: 202 len: 0 microversion: 2.1 time: 0.022817 Oct 12 02:40:32.085505 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 551/1100] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 12 02:40:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-474987065 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:40:39.892163 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:39.892805 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:39.899727 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:39.900207 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:39.900645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:39.901348 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:39.901798 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:39.902215 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:39.924438 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-14094828-0c9e-4ac3-b9a4-2305e48192bb tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/ips" status: 200 len: 146 microversion: 2.1 time: 0.036192 Oct 12 02:40:39.925010 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 550/1101] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 12 02:40:39 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/ips => generated 146 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:41.252759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:41.256885 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e845ab00-6f96-4322-aa0d-36a7c71ade02"}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:40:41.259154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:41.266812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:41.267299 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:41.267729 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:41.268423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:41.268918 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:41.269345 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:41.528519 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5bfcdf03-7537-407f-8150-94072c2f326b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.276973 Oct 12 02:40:41.529150 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 552/1102] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:40:41 2022] POST /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments => generated 194 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:47.997701 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:47.999970 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:48.006008 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:48.006486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:48.007417 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:48.008098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:48.008549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:48.008994 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:48.038593 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-96ee7717-fb55-4e3a-9b1c-e19355257264 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044569 Oct 12 02:40:48.039399 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 551/1103] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:40:47 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:48.051045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:48.051619 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:48.058082 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:48.058769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:48.059202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:48.059894 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:48.060343 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:48.060792 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:48.090030 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4444dae6-52b5-4981-a1db-fc4f473590db tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/e845ab00-6f96-4322-aa0d-36a7c71ade02" status: 200 len: 194 microversion: 2.1 time: 0.043626 Oct 12 02:40:48.090697 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 553/1104] 10.222.0.23 () {58 vars in 1413 bytes} [Wed Oct 12 02:40:48 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/e845ab00-6f96-4322-aa0d-36a7c71ade02 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:49.421929 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f3d93295-7312-4785-a3ec-e86f83ad0348"}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:40:49.424220 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:49.431143 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:49.431616 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:49.432042 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:49.432737 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:49.433189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:49.433607 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:49.693007 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ed1c93e2-994f-49ba-96fa-44c2a6fa1379 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "POST /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.274764 Oct 12 02:40:49.693603 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 552/1105] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 12 02:40:49 2022] POST /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments => generated 194 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:57.215550 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:57.217801 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:57.224195 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:57.224967 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:57.225411 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:57.226096 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:57.226550 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:57.226969 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:57.257023 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bbb92627-2773-433a-81b1-9411b4f0b863 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.045287 Oct 12 02:40:57.257578 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 554/1106] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:40:57 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments => generated 371 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:57.401737 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:57.402245 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:57.408435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:57.408945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:57.409379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:57.410063 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:57.410513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:57.410937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:57.439860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a0e21990-6054-4bd6-965e-96abe245385c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/e845ab00-6f96-4322-aa0d-36a7c71ade02" status: 200 len: 194 microversion: 2.1 time: 0.043142 Oct 12 02:40:57.440420 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 553/1107] 10.222.0.23 () {58 vars in 1413 bytes} [Wed Oct 12 02:40:57 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/e845ab00-6f96-4322-aa0d-36a7c71ade02 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:57.450628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:57.451116 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:57.457105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:57.457584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:57.458011 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:57.458720 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:57.459166 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:57.459578 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:57.538590 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lazy-loading 'services' on Instance uuid 50e73cb0-a161-4344-be58-ebf9dc84c4a2 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:40:57.609364 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-23fc245d-db96-4e01-a67d-6f79f1d1cb8c tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/e845ab00-6f96-4322-aa0d-36a7c71ade02" status: 202 len: 0 microversion: 2.1 time: 0.162394 Oct 12 02:40:57.609955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 555/1108] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:40:57 2022] DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/e845ab00-6f96-4322-aa0d-36a7c71ade02 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:40:58.727809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:58.728298 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:58.730682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=17 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:58.735867 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:58.736371 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:58.736836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:58.737516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:58.737967 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:58.738384 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:58.742076 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:58.769348 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0000cb75-7f11-4a7c-a0a3-8c011a484a09 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/f3d93295-7312-4785-a3ec-e86f83ad0348" status: 200 len: 194 microversion: 2.1 time: 0.045749 Oct 12 02:40:58.770087 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 554/1109] 10.222.0.23 () {58 vars in 1413 bytes} [Wed Oct 12 02:40:58 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/f3d93295-7312-4785-a3ec-e86f83ad0348 => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:40:58.781458 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:40:58.782037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:40:58.784026 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=23,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:40:58.789398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:58.789902 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:58.790332 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:58.791045 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:40:58.791494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:40:58.791914 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:40:58.874666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.objects.instance [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lazy-loading 'services' on Instance uuid 50e73cb0-a161-4344-be58-ebf9dc84c4a2 {{(pid=118864) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 12 02:40:58.947355 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a51f64bb-20ff-4737-9fa3-315eeddc968e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/f3d93295-7312-4785-a3ec-e86f83ad0348" status: 202 len: 0 microversion: 2.1 time: 0.170472 Oct 12 02:40:58.947932 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 556/1110] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 12 02:40:58 2022] DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments/f3d93295-7312-4785-a3ec-e86f83ad0348 => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:41:00.120847 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:00.123094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:00.129449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.129934 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.130369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.131059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.131516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.131939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.160368 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-cf4bb98a-044f-4158-a5c6-d64dfbd5eb8d tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043952 Oct 12 02:41:00.160940 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 555/1111] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:41:00 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:41:00.274718 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:00.277495 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:00.283645 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.284130 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.284567 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.285292 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.285747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.286171 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.316834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-5357d763-773c-4dd4-bc08-118991f39bc6 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046954 Oct 12 02:41:00.317382 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 557/1112] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 12 02:41:00 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 12 02:41:00.379388 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:00.379879 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:00.386231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.386711 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.387140 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.387824 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.388274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.388712 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.413017 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:41:00.486181 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ad64d5a5-c4a0-4c4f-a08b-e398c7a456fa tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 204 len: 0 microversion: 2.1 time: 0.110203 Oct 12 02:41:00.486842 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 556/1113] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:41:00 2022] DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:41:00.495016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:00.495625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:00.501461 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.501942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.502396 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.503054 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:00.503504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:00.503923 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:00.629851 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:41:00.637982 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-682aeb68-b50e-4979-a71e-f5875662c69e tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1674 microversion: 2.1 time: 0.146855 Oct 12 02:41:00.638545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 558/1114] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:41:00 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1674 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:41:01.662331 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:01.663086 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:01.670274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:01.670760 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:01.671199 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:01.671924 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:01.672382 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:01.672839 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:01.805046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:41:01.812597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fa1deffb-bd25-4d2b-8557-2be9db893b50 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1674 microversion: 2.1 time: 0.154666 Oct 12 02:41:01.813192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 557/1115] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:41:01 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1674 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:41:02.835401 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:02.836011 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:02.842991 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:02.843482 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:02.843962 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:02.844661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:02.845149 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:02.845572 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:02.979231 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea7aceaca15a47f89c674c237b0cacae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '297b4af1911c49f3b7b01c35236f4dae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:41:02.986990 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fff9245c-5143-4cb3-9545-480e41e2be2f tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 200 len: 1674 microversion: 2.1 time: 0.155434 Oct 12 02:41:02.987549 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 559/1116] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:41:02 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 1674 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:41:03.229756 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-fa4b4096-2c8d-4b00-a574-785749d394ff req-dd10ef43-6594-4333-95fd-f96a441ed953 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "50e73cb0-a161-4344-be58-ebf9dc84c4a2", "name": "network-vif-deleted", "tag": "e1dfc227-256b-42f2-8bbc-5b489c6f72d3"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:41:03.240541 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-fa4b4096-2c8d-4b00-a574-785749d394ff req-dd10ef43-6594-4333-95fd-f96a441ed953 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:03.241084 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-fa4b4096-2c8d-4b00-a574-785749d394ff req-dd10ef43-6594-4333-95fd-f96a441ed953 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:03.241513 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-fa4b4096-2c8d-4b00-a574-785749d394ff req-dd10ef43-6594-4333-95fd-f96a441ed953 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:03.253279 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-fa4b4096-2c8d-4b00-a574-785749d394ff req-dd10ef43-6594-4333-95fd-f96a441ed953 service nova] Creating event network-vif-deleted:e1dfc227-256b-42f2-8bbc-5b489c6f72d3 for instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 on hv-cinder-84914 Oct 12 02:41:03.258829 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-fa4b4096-2c8d-4b00-a574-785749d394ff req-dd10ef43-6594-4333-95fd-f96a441ed953 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033754 Oct 12 02:41:03.261178 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 558/1117] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:41:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:41:04.011258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:04.011883 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:04.018056 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:04.018537 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:04.018972 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:04.019663 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:04.020109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:04.020528 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:04.034047 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] HTTP exception thrown: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Request method failure captured: Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 HTTP/1.1 Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRihb5UIAeD_NP7kC_ZFsPo1l7SWXoUv7g_UGBxRAHC7_uJJaiN2yn8aP0n-5xK3fEnWR67d5TD0kTMIK-k3kM64hiafEzYEyLTBsbpN7XKlozvmm67R__jnUzX1c2w6EVsnn3phUUBw0Ddk02eXi21nqFeep6_L6zUUZHb5WYXF6xHw Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:41:04.035696 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_297b4af1911c49f3b7b01c35236f4dae", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: ea7aceaca15a47f89c674c237b0cacae Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:04.037288 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '50e73cb0-a161-4344-be58-ebf9dc84c4a2'} Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:41:04.038248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:04.039570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:04.040795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Returning 404 to user: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:41:04.040795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e2d9301b-b65a-465f-9633-e22ae27b7866 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 404 len: 111 microversion: 2.1 time: 0.030318 Oct 12 02:41:04.040795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 560/1118] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:41:04 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:41:04.047414 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-bac667cd-2d09-42f2-9b99-c553802dbf4b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '.version_select at 0x7f59d43bd5e0>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:04.066588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-bac667cd-2d09-42f2-9b99-c553802dbf4b tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1240949202" status: 202 len: 0 microversion: 2.1 time: 0.023435 Oct 12 02:41:04.067364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 559/1119] 10.222.0.23 () {58 vars in 1221 bytes} [Wed Oct 12 02:41:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1240949202 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 12 02:41:06.938118 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:06.938605 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:06.945120 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:06.945601 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:06.946029 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:06.946711 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:06.947155 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:06.947570 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:06.962647 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] HTTP exception thrown: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Request method failure captured: Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 HTTP/1.1 Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRihb5UIAeD_NP7kC_ZFsPo1l7SWXoUv7g_UGBxRAHC7_uJJaiN2yn8aP0n-5xK3fEnWR67d5TD0kTMIK-k3kM64hiafEzYEyLTBsbpN7XKlozvmm67R__jnUzX1c2w6EVsnn3phUUBw0Ddk02eXi21nqFeep6_L6zUUZHb5WYXF6xHw Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:41:06.964252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_297b4af1911c49f3b7b01c35236f4dae", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: ea7aceaca15a47f89c674c237b0cacae Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:06.965880 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '50e73cb0-a161-4344-be58-ebf9dc84c4a2'} Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:41:06.966823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:06.968135 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:06.969527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Returning 404 to user: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:41:06.969527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-89f2e21d-01d2-4af2-a59c-b90059d315a2 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 404 len: 111 microversion: 2.1 time: 0.031622 Oct 12 02:41:06.969527 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 561/1120] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:41:06 2022] DELETE /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:41:06.975519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:06.976249 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: 50e73cb0-a161-4344-be58-ebf9dc84c4a2] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:06.982933 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:06.983418 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:06.983848 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:06.984532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:06.985036 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:06.985459 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:07.003741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] HTTP exception thrown: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Request method failure captured: Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 HTTP/1.1 Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRihb5UIAeD_NP7kC_ZFsPo1l7SWXoUv7g_UGBxRAHC7_uJJaiN2yn8aP0n-5xK3fEnWR67d5TD0kTMIK-k3kM64hiafEzYEyLTBsbpN7XKlozvmm67R__jnUzX1c2w6EVsnn3phUUBw0Ddk02eXi21nqFeep6_L6zUUZHb5WYXF6xHw Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:41:07.005454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_297b4af1911c49f3b7b01c35236f4dae", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: ea7aceaca15a47f89c674c237b0cacae Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:07.007094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': '50e73cb0-a161-4344-be58-ebf9dc84c4a2'} Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:41:07.008222 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. Oct 12 02:41:07.009785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:07.012005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Returning 404 to user: Instance 50e73cb0-a161-4344-be58-ebf9dc84c4a2 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:41:07.012005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b6c90da1-e78b-4190-a699-9aed09591128 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2" status: 404 len: 111 microversion: 2.1 time: 0.035879 Oct 12 02:41:07.012005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 560/1121] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:41:06 2022] GET /compute/v2.1/servers/50e73cb0-a161-4344-be58-ebf9dc84c4a2 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:41:08.144277 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:08.144785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:08.151305 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:08.151785 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:08.152217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:08.152947 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:08.153406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:08.153831 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:08.167514 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] HTTP exception thrown: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Request method failure captured: Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: DELETE /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 HTTP/1.1 Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRihb5UIAeD_NP7kC_ZFsPo1l7SWXoUv7g_UGBxRAHC7_uJJaiN2yn8aP0n-5xK3fEnWR67d5TD0kTMIK-k3kM64hiafEzYEyLTBsbpN7XKlozvmm67R__jnUzX1c2w6EVsnn3phUUBw0Ddk02eXi21nqFeep6_L6zUUZHb5WYXF6xHw Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:41:08.169062 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_297b4af1911c49f3b7b01c35236f4dae", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: ea7aceaca15a47f89c674c237b0cacae Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.170699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'f5970385-f587-4e45-b2ca-9d5913f90396'} Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:41:08.171592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.173109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.174934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Returning 404 to user: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:41:08.174934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-a1da2aeb-76e9-4d9c-8bfa-a0f923f5fb81 tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "DELETE /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 404 len: 111 microversion: 2.1 time: 0.030043 Oct 12 02:41:08.174934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 562/1122] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:41:08 2022] DELETE /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:41:08.178485 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:41:08.179098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] [instance: f5970385-f587-4e45-b2ca-9d5913f90396] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:41:08.186087 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:08.186563 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:08.186995 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:08.187676 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:41:08.188127 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:41:08.188547 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:41:08.201463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] HTTP exception thrown: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Request method failure captured: Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 HTTP/1.1 Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRihb5UIAeD_NP7kC_ZFsPo1l7SWXoUv7g_UGBxRAHC7_uJJaiN2yn8aP0n-5xK3fEnWR67d5TD0kTMIK-k3kM64hiafEzYEyLTBsbpN7XKlozvmm67R__jnUzX1c2w6EVsnn3phUUBw0Ddk02eXi21nqFeep6_L6zUUZHb5WYXF6xHw Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:41:08.202945 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/297b4af1911c49f3b7b01c35236f4dae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_297b4af1911c49f3b7b01c35236f4dae", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 297b4af1911c49f3b7b01c35236f4dae Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1909094045 Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: ea7aceaca15a47f89c674c237b0cacae Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-AttachVolumeTestJSON-1909094045-project Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.204580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'f5970385-f587-4e45-b2ca-9d5913f90396'} Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:41:08.205490 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. Oct 12 02:41:08.206935 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:41:08.208335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] Returning 404 to user: Instance f5970385-f587-4e45-b2ca-9d5913f90396 could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:41:08.208335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-43614896-0bbf-4e24-8f16-933ef5d665df tempest-AttachVolumeTestJSON-1909094045 tempest-AttachVolumeTestJSON-1909094045-project] 10.222.0.23 "GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396" status: 404 len: 111 microversion: 2.1 time: 0.029588 Oct 12 02:41:08.208335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 561/1123] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:41:08 2022] GET /compute/v2.1/servers/f5970385-f587-4e45-b2ca-9d5913f90396 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:42:18.175134 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=13 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:18.176321 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=65,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:18.177828 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:42:18.177998 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Wed Oct 12 02:42:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs (ip 10.222.0.23) !!! Oct 12 02:42:18.356038 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-4e3bfded-2324-46fa-86e4-e218088a86e4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Action: 'create', calling method: .version_select at 0x7f59d35ccaf0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-789738130", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPMp+I0jAcCIZPIdwYuPEfWq3w8RyFXWS/Amd+FJXiZpBPB2Rd2ZwgDew7uzCfhtGM23JjggqhIzgdWEr0+2TAXL8VOZqb+wk5D40Fy8U5c0rSCx7oO+kOtxQj5brYCXGw=="}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:42:18.372418 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4e3bfded-2324-46fa-86e4-e218088a86e4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:42:18.377028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-4e3bfded-2324-46fa-86e4-e218088a86e4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for user e184afaa0abd4e8eb1bc641167dc0b9b and project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'key_pairs'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:42:18.399719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-4e3bfded-2324-46fa-86e4-e218088a86e4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.226214 Oct 12 02:42:18.400329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 563/1124] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 12 02:42:18 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:42:33.811720 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=21,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:33.812491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=37,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:33.817370 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-828383411", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-789738130", "security_groups": [{"name": "tempest-secgroup-smoke-650875271"}], "block_device_mapping_v2": [{"uuid": "a51de88c-47f6-4535-a94e-0e8f96886b44", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c8f4d944-2283-4081-9e18-98805e4df4db"}]}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:42:33.931276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] validate_networks() for [('c8f4d944-2283-4081-9e18-98805e4df4db', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:42:34.289103 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:42:34.289525 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:42:34.289998 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:42:34.290405 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:42:34.290816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:42:34.299949 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:34.300429 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:34.300906 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:34.301645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:34.302098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:34.302519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:34.319789 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:42:34.323938 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for user e184afaa0abd4e8eb1bc641167dc0b9b and project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:42:34.380090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:42:34.382230 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a51de88c-47f6-4535-a94e-0e8f96886b44',volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:42:34.515079 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-46c64165-a9c6-4bdb-9c3b-cd371f45cdc4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.705185 Oct 12 02:42:34.515725 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 562/1125] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:42:33 2022] POST /compute/v2.1/servers => generated 405 bytes in 707 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:42:34.524666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=4 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:34.526039 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:34.531163 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e9dcba77-e1bf-44be-a0fd-ecc644fbcf38 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:34.543701 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e9dcba77-e1bf-44be-a0fd-ecc644fbcf38 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:34.610207 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e9dcba77-e1bf-44be-a0fd-ecc644fbcf38 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:34.617812 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e9dcba77-e1bf-44be-a0fd-ecc644fbcf38 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1093 microversion: 2.1 time: 0.094834 Oct 12 02:42:34.618378 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 564/1126] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:34 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1093 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:35.641684 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:35.642300 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:35.648282 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:35.648796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:35.649231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:35.649913 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:35.650369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:35.650787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:35.722884 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:35.730812 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7f1d837d-4182-4f7f-9fc4-ec0579cf04ff tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1199 microversion: 2.1 time: 0.094431 Oct 12 02:42:35.732801 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 563/1127] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:35 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1199 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:36.755545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:36.756673 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:36.763072 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:36.763555 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:36.763985 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:36.764666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:36.765311 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:36.765730 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:36.899906 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:36.908310 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fd52e1a5-1263-4f69-afbd-7c3abfe314c3 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.156641 Oct 12 02:42:36.909179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 565/1128] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:36 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:37.165845 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-25fecfff-de03-40b4-80ae-ff38b43214c8 req-ddf82f99-8d10-4ea4-bb39-e76c6a8a5e28 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b42a22ef-f44f-4e88-88ca-2ea755acca2a", "tag": "3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:42:37.178960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-25fecfff-de03-40b4-80ae-ff38b43214c8 req-ddf82f99-8d10-4ea4-bb39-e76c6a8a5e28 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:37.183579 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-25fecfff-de03-40b4-80ae-ff38b43214c8 req-ddf82f99-8d10-4ea4-bb39-e76c6a8a5e28 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:37.184062 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-25fecfff-de03-40b4-80ae-ff38b43214c8 req-ddf82f99-8d10-4ea4-bb39-e76c6a8a5e28 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:37.198401 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-25fecfff-de03-40b4-80ae-ff38b43214c8 req-ddf82f99-8d10-4ea4-bb39-e76c6a8a5e28 service nova] Creating event network-changed:3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a for instance b42a22ef-f44f-4e88-88ca-2ea755acca2a on hv-cinder-84914 Oct 12 02:42:37.205661 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-25fecfff-de03-40b4-80ae-ff38b43214c8 req-ddf82f99-8d10-4ea4-bb39-e76c6a8a5e28 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044421 Oct 12 02:42:37.206587 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 564/1129] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:42:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:42:37.931854 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:37.933042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:37.940100 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:37.940592 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:37.941056 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:37.941766 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:37.942224 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:37.942652 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:38.073418 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:38.081025 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-ec707ba9-56c7-43c6-9124-09a2d6b119b4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.153306 Oct 12 02:42:38.081584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 566/1130] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:37 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:39.105419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:39.106014 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:39.112776 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:39.113264 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:39.113702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:39.114397 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:39.114860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:39.115285 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:39.251525 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:39.259204 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4cedd04f-d359-4f33-8a29-35cb22005b08 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.159024 Oct 12 02:42:39.259770 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 565/1131] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:39 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:40.281022 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:40.281634 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:40.287962 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:40.288445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:40.288912 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:40.289598 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:40.290048 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:40.290471 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:40.422893 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:40.430504 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8090335d-c129-4173-bd29-2ef57aa46901 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.153447 Oct 12 02:42:40.431074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 567/1132] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:40 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:41.453741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:41.454350 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:41.461741 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:41.462225 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:41.462658 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:41.463343 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:41.463792 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:41.464292 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:41.607487 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:41.615584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-7f7c1f62-9962-498a-a4a6-f62523e73e20 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.166417 Oct 12 02:42:41.616144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 566/1133] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:41 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:42.638940 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:42.639551 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:42.646486 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:42.646968 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:42.647409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:42.648092 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:42.648546 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:42.648994 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:42.774456 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:42.782517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e83244dc-a2c3-41b2-92c2-1f8d29084c5d tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.147446 Oct 12 02:42:42.783080 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 568/1134] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:42 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:43.805517 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:43.806132 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:43.813052 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:43.813539 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:43.813972 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:43.814657 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:43.815111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:43.815535 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:43.944512 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:43.952160 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-de0ff1d3-50db-486d-90e8-12e54bcbc793 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.150522 Oct 12 02:42:43.952753 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 567/1135] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:43 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:44.974656 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:44.975265 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:44.977850 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:44.983235 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:44.983718 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:44.984153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:44.984874 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:44.985329 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:44.985752 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:45.115565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:45.123367 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-67dbf7c7-5f89-4873-9d7b-ff7c54c4680c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.152534 Oct 12 02:42:45.123929 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 569/1136] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:44 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:42:46.146316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:46.146960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:46.148990 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:46.149645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=6 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:42:46.156739 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:46.157221 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:46.157647 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:46.158324 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:46.158775 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:46.159189 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:46.290065 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:46.297559 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-ea27fa6d-3406-4cb6-9ffd-43790e82e18c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.155311 Oct 12 02:42:46.298111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 568/1137] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:46 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:47.320742 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:47.321378 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:47.327693 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:47.328177 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:47.328608 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:47.329315 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:47.329768 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:47.330188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:47.462392 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:47.470848 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-b55708f8-0cbb-419f-b746-143add24f947 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.153877 Oct 12 02:42:47.471613 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 570/1138] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:47 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:47.648374 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-70c2b341-05e0-414c-879b-18d7f9c17657 req-78c34211-bcf3-43cd-8322-5232f88874cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b42a22ef-f44f-4e88-88ca-2ea755acca2a", "name": "network-vif-plugged", "status": "completed", "tag": "3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:42:47.658118 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-70c2b341-05e0-414c-879b-18d7f9c17657 req-78c34211-bcf3-43cd-8322-5232f88874cb service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:47.658595 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-70c2b341-05e0-414c-879b-18d7f9c17657 req-78c34211-bcf3-43cd-8322-5232f88874cb service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:47.659016 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-70c2b341-05e0-414c-879b-18d7f9c17657 req-78c34211-bcf3-43cd-8322-5232f88874cb service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:47.669601 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-70c2b341-05e0-414c-879b-18d7f9c17657 req-78c34211-bcf3-43cd-8322-5232f88874cb service nova] Creating event network-vif-plugged:3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a for instance b42a22ef-f44f-4e88-88ca-2ea755acca2a on hv-cinder-84914 Oct 12 02:42:47.675028 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-70c2b341-05e0-414c-879b-18d7f9c17657 req-78c34211-bcf3-43cd-8322-5232f88874cb service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092834 Oct 12 02:42:47.675630 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 569/1139] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:42:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:42:48.493484 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:48.494085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:48.500437 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:48.500954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:48.501384 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:48.502060 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:48.502507 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:48.502922 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:48.630064 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:48.637601 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-bf6ca176-616a-4d49-baa2-b8f6a92c6abb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1264 microversion: 2.1 time: 0.147950 Oct 12 02:42:48.638157 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 571/1140] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:48 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1264 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:49.667303 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:49.667999 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:49.674685 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:49.675161 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:49.675594 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:49.676274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:49.676743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:49.677164 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:49.809922 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:49.817523 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-20a5f004-a4db-4925-bcbb-c1e0f4e7527b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1452 microversion: 2.1 time: 0.154873 Oct 12 02:42:49.818088 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 570/1141] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:49 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1452 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:49.839739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:42:49.840352 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:42:49.847073 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:49.847621 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:49.848075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:49.848795 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:49.849301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:49.849802 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:49.981569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:42:49.989247 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-da9ee77a-e4fd-4225-8e1a-79ba63f7b84c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1452 microversion: 2.1 time: 0.153388 Oct 12 02:42:49.989815 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 572/1142] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:42:49 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1452 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:42:51.895555 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-158da804-5c58-4cd0-8790-acffc895a1ff req-c2b1988d-7fe4-403f-9cbf-7ddfa442d272 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b42a22ef-f44f-4e88-88ca-2ea755acca2a", "tag": "3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:42:51.904942 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-158da804-5c58-4cd0-8790-acffc895a1ff req-c2b1988d-7fe4-403f-9cbf-7ddfa442d272 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:42:51.905415 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-158da804-5c58-4cd0-8790-acffc895a1ff req-c2b1988d-7fe4-403f-9cbf-7ddfa442d272 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:42:51.905836 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-158da804-5c58-4cd0-8790-acffc895a1ff req-c2b1988d-7fe4-403f-9cbf-7ddfa442d272 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:42:51.916475 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-158da804-5c58-4cd0-8790-acffc895a1ff req-c2b1988d-7fe4-403f-9cbf-7ddfa442d272 service nova] Creating event network-changed:3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a for instance b42a22ef-f44f-4e88-88ca-2ea755acca2a on hv-cinder-84914 Oct 12 02:42:51.921239 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-158da804-5c58-4cd0-8790-acffc895a1ff req-c2b1988d-7fe4-403f-9cbf-7ddfa442d272 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084034 Oct 12 02:42:51.921991 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 571/1143] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:42:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:44:45.923125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=20 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:44:45.924413 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=48 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:44:45.926007 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 12 02:44:45.926211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Wed Oct 12 02:44:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a (ip 10.222.0.23) !!! Oct 12 02:44:45.928356 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:44:45.928540 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Wed Oct 12 02:44:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a (ip 10.222.0.23) !!! Oct 12 02:44:45.941226 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:45.941722 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:45.953623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:45.954105 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:45.954535 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:45.955225 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:45.955674 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:45.956097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:45.984576 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Going to try to terminate instance {{(pid=118864) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:44:46.057582 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-39c1a77d-19f2-4586-8c79-f02f64170ecd tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "DELETE /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 204 len: 0 microversion: 2.1 time: 0.136183 Oct 12 02:44:46.058154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 573/1144] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:44:45 2022] DELETE /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:44:46.066319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:44:46.067149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=44 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:44:46.068956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 12 02:44:46.069111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Wed Oct 12 02:44:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a (ip 10.222.0.23) !!! Oct 12 02:44:46.071441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:44:46.071566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Wed Oct 12 02:44:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a (ip 10.222.0.23) !!! Oct 12 02:44:46.084509 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:46.085228 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:46.093035 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:46.093514 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:46.094037 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:46.094705 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:46.095150 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:46.095566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:46.275724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:46.283530 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-a370d339-357f-4ddd-8b38-bd0ace1c17e6 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1577 microversion: 2.1 time: 0.220802 Oct 12 02:44:46.284109 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 572/1145] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:46 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1577 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:47.307234 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:47.307927 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:47.314283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:47.314760 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:47.315187 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:47.315864 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:47.316336 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:47.316777 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:47.496142 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:47.503738 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-efcef1c2-1eb1-43b6-8b35-6d3bb7da8f0e tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1577 microversion: 2.1 time: 0.200723 Oct 12 02:44:47.504297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 574/1146] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:47 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1577 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:48.527435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:48.528048 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:48.534441 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:48.534937 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:48.535371 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:48.536077 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:48.536531 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:48.537001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:48.782823 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:48.790986 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2ab52efb-1a87-466e-a31e-85fdcf2b04a2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 200 len: 1510 microversion: 2.1 time: 0.267600 Oct 12 02:44:48.791562 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 573/1147] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:48 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 1510 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:48.850769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-36fd4588-0763-42a4-b17f-f9954f6545ca req-11feb48a-5539-4150-9784-d7969ae6704b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b42a22ef-f44f-4e88-88ca-2ea755acca2a", "name": "network-vif-deleted", "tag": "3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:44:48.861057 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-36fd4588-0763-42a4-b17f-f9954f6545ca req-11feb48a-5539-4150-9784-d7969ae6704b service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:48.861552 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-36fd4588-0763-42a4-b17f-f9954f6545ca req-11feb48a-5539-4150-9784-d7969ae6704b service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:48.862053 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-36fd4588-0763-42a4-b17f-f9954f6545ca req-11feb48a-5539-4150-9784-d7969ae6704b service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:48.875173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-36fd4588-0763-42a4-b17f-f9954f6545ca req-11feb48a-5539-4150-9784-d7969ae6704b service nova] Creating event network-vif-deleted:3d1c5f6f-6ba8-4b9d-ac68-58a183e1c23a for instance b42a22ef-f44f-4e88-88ca-2ea755acca2a on hv-cinder-84914 Oct 12 02:44:48.881041 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-36fd4588-0763-42a4-b17f-f9954f6545ca req-11feb48a-5539-4150-9784-d7969ae6704b service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033892 Oct 12 02:44:48.881719 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 575/1148] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:44:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:44:49.814921 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:49.815591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:49.822636 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:49.823082 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:49.823600 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:49.824187 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:49.824637 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:49.825086 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:49.838239 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] HTTP exception thrown: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Request method failure captured: Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a HTTP/1.1 Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRimE-Fp-G6xyxkrLZMxoeKfW_1MJtImcqjDq-YhjXhXXA2s1mtCDjmw2-NvvZt4N4kAzXZ9UzDGw9kJmdGQ6BNVqdDB8F3nY025Vf1v5qzOsh372asUi60wK4h3R4kxjVD_CelLyPtN84tvTp2q0cNN1wqrNK1o7M4dPpOkH-bY7kpo Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:44:49.839875 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_0420b1fe8ff5451dba85de3001a3932f", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-TestVolumeBootPattern-608330013 Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: e184afaa0abd4e8eb1bc641167dc0b9b Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:44:49.841787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'b42a22ef-f44f-4e88-88ca-2ea755acca2a'} Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:44:49.842901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:44:49.844537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:44:49.846041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Returning 404 to user: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:44:49.846041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6e637a63-89b0-48b2-8480-520df4861d48 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 404 len: 111 microversion: 2.1 time: 0.030331 Oct 12 02:44:49.846041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 574/1149] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:49 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:44:49.890450 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-509771881", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-789738130", "security_groups": [{"name": "tempest-secgroup-smoke-650875271"}], "block_device_mapping_v2": [{"uuid": "a51de88c-47f6-4535-a94e-0e8f96886b44", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c8f4d944-2283-4081-9e18-98805e4df4db"}]}} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:44:50.004814 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.network.neutron [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] validate_networks() for [('c8f4d944-2283-4081-9e18-98805e4df4db', None, None, None, None, None)] {{(pid=118864) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:44:50.413475 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Flavor limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:44:50.413891 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Image limits 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:44:50.414370 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Flavor pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:44:50.414768 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Image pref 0:0:0 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:44:50.415173 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.virt.hardware [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118864) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:44:50.423653 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:50.424125 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:50.424553 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:50.425310 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:50.425763 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:50.426248 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:50.442250 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:44:50.446179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.quota [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for user e184afaa0abd4e8eb1bc641167dc0b9b and project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'ram', 'cores', 'instances'} {{(pid=118864) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:44:50.502556 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Going to run 1 instances... {{(pid=118864) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:44:50.504796 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a51de88c-47f6-4535-a94e-0e8f96886b44',volume_size=None,volume_type=None)] {{(pid=118864) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:44:50.629815 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-526b3fbe-89de-4945-8573-fd41f8946b4b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.744076 Oct 12 02:44:50.630450 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 576/1150] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:44:49 2022] POST /compute/v2.1/servers => generated 405 bytes in 746 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:44:50.641481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-c8080a08-63b8-43bb-8d40-99bf1b98e47c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:50.642175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-c8080a08-63b8-43bb-8d40-99bf1b98e47c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:50.699309 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-c8080a08-63b8-43bb-8d40-99bf1b98e47c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:50.706481 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-c8080a08-63b8-43bb-8d40-99bf1b98e47c tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1093 microversion: 2.1 time: 0.068717 Oct 12 02:44:50.707866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 575/1151] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:50 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1093 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:51.730283 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:51.731017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:51.737513 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:51.738512 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:51.738948 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:51.739648 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:51.740097 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:51.740772 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:51.803217 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:51.810782 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-df69ce3d-3207-43a3-8534-6d5d3ab0e0cb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1209 microversion: 2.1 time: 0.085081 Oct 12 02:44:51.811368 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 577/1152] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:51 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1209 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:52.834451 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:52.835059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:52.842267 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:52.842746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:52.843175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:52.843860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:52.844304 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:52.844733 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:53.042314 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:53.049974 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e9f08d7e-c302-4d60-9bd9-699b4a437d64 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.219881 Oct 12 02:44:53.051044 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 576/1153] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:52 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:53.119656 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-098e185d-3e1b-4ae7-a883-e7bcd9a62c46 req-22127006-adc6-4752-9af5-795b51480f32 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "269ae487-b0f5-4513-a5a9-baa48cc9883a", "tag": "6a932fdd-a6eb-4530-952f-21adba6785ea"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:44:53.130148 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-098e185d-3e1b-4ae7-a883-e7bcd9a62c46 req-22127006-adc6-4752-9af5-795b51480f32 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:53.130601 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-098e185d-3e1b-4ae7-a883-e7bcd9a62c46 req-22127006-adc6-4752-9af5-795b51480f32 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:53.131087 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-098e185d-3e1b-4ae7-a883-e7bcd9a62c46 req-22127006-adc6-4752-9af5-795b51480f32 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:53.142066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-098e185d-3e1b-4ae7-a883-e7bcd9a62c46 req-22127006-adc6-4752-9af5-795b51480f32 service nova] Creating event network-changed:6a932fdd-a6eb-4530-952f-21adba6785ea for instance 269ae487-b0f5-4513-a5a9-baa48cc9883a on hv-cinder-84914 Oct 12 02:44:53.146799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-098e185d-3e1b-4ae7-a883-e7bcd9a62c46 req-22127006-adc6-4752-9af5-795b51480f32 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031259 Oct 12 02:44:53.147391 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 578/1154] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:44:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:44:54.073538 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:54.074144 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:54.080409 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:54.080874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:54.081307 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:54.081993 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:54.082448 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:54.082869 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:54.289454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:54.297018 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-70acb549-fada-43fc-b818-968818f8076f tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.227310 Oct 12 02:44:54.297584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 577/1155] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:54 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:55.321529 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:55.322250 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:55.328310 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:55.328820 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:55.329257 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:55.329944 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:55.330398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:55.330823 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:55.463716 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:55.471231 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-9c7576a2-7f09-411d-b4c0-4d1bd29844d4 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.154411 Oct 12 02:44:55.471794 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 579/1156] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:55 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:56.493950 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:56.494554 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:56.500985 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:56.501472 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:56.501932 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:56.502618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:56.503099 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:56.503518 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:56.629171 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:56.636721 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-b7c2a153-07b5-4f6a-96cc-ecd834c6b5ea tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.146729 Oct 12 02:44:56.637274 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 578/1157] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:56 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:57.659448 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:57.660028 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:57.666404 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:57.667153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:57.667615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:57.668303 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:57.668786 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:57.669216 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:57.798221 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:57.806317 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-689d69b7-ff8a-47e2-8cd9-12016c7f186a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.150643 Oct 12 02:44:57.806881 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 580/1158] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:57 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:44:58.829908 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:44:58.830516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:44:58.837400 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:58.837877 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:58.838308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:58.838989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:44:58.839435 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:44:58.839853 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:44:58.971217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:44:58.978718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-77e7e30c-e767-4faa-ae5b-95b999e377db tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.153112 Oct 12 02:44:58.979572 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 579/1159] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:58 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:45:00.001579 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:45:00.002188 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:45:00.009974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:00.010460 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:00.010894 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:00.011642 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:00.012106 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:00.012529 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:00.142749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:45:00.150844 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-096f0f54-8d6c-45fc-bb7c-99fede2da315 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.153183 Oct 12 02:45:00.151407 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 581/1160] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:44:59 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:45:01.174983 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:45:01.175599 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:45:01.178568 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:45:01.184111 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:01.184596 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:01.185055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:01.185746 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:01.186201 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:01.186629 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:01.314889 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:45:01.322425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e6017cf4-d2b0-4475-9c95-0c25740f54dc tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.151285 Oct 12 02:45:01.322989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 580/1161] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:45:01 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:45:02.344419 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:45:02.345052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:45:02.347004 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:45:02.347664 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=6 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:45:02.353910 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:02.354662 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:02.355094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:02.355781 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:02.356234 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:02.356734 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:02.487410 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:45:02.495372 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0d8d96e4-93bd-4f48-9512-d9b4d0daf6c7 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.154772 Oct 12 02:45:02.495946 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 582/1162] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:45:02 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:45:03.518455 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:45:03.519068 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:45:03.526005 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:03.526484 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:03.526917 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:03.527598 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:03.528049 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:03.528499 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:03.657218 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:45:03.664762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-f89bae28-6087-44c2-9548-b3fc0c340ba0 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.150187 Oct 12 02:45:03.665323 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 581/1163] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:45:03 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:45:04.079126 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-6c0749eb-85c4-4f6a-9051-35a77e71e853 req-c065d877-dad3-473b-a268-479147beaa0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "269ae487-b0f5-4513-a5a9-baa48cc9883a", "name": "network-vif-plugged", "status": "completed", "tag": "6a932fdd-a6eb-4530-952f-21adba6785ea"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:45:04.089266 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-6c0749eb-85c4-4f6a-9051-35a77e71e853 req-c065d877-dad3-473b-a268-479147beaa0c service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:04.089747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-6c0749eb-85c4-4f6a-9051-35a77e71e853 req-c065d877-dad3-473b-a268-479147beaa0c service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:04.090175 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-6c0749eb-85c4-4f6a-9051-35a77e71e853 req-c065d877-dad3-473b-a268-479147beaa0c service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:04.101406 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-6c0749eb-85c4-4f6a-9051-35a77e71e853 req-c065d877-dad3-473b-a268-479147beaa0c service nova] Creating event network-vif-plugged:6a932fdd-a6eb-4530-952f-21adba6785ea for instance 269ae487-b0f5-4513-a5a9-baa48cc9883a on hv-cinder-84914 Oct 12 02:45:04.106662 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-6c0749eb-85c4-4f6a-9051-35a77e71e853 req-c065d877-dad3-473b-a268-479147beaa0c service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031258 Oct 12 02:45:04.107294 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 583/1164] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:45:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:45:04.687535 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:45:04.688072 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:45:04.695292 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:04.695775 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:04.696213 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:04.696932 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:04.697388 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:04.697809 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:04.826738 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:45:04.834324 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-3e452344-b84e-4db0-835c-1d94a6890197 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1264 microversion: 2.1 time: 0.150865 Oct 12 02:45:04.834888 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 582/1165] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:45:04 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:45:05.857871 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:45:05.858934 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:45:05.866211 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:05.866687 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:05.867117 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:05.867793 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:05.868241 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:05.868657 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:05.997163 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:45:06.004841 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-0d3811c6-d8b5-41b2-9ed0-76fdd6cfdc61 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1451 microversion: 2.1 time: 0.151934 Oct 12 02:45:06.005417 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 584/1166] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:45:05 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1451 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:45:06.029591 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:45:06.030196 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:45:06.037265 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:06.037748 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:06.038178 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:06.038858 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:06.039335 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:06.039756 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:06.170529 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:45:06.178106 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-78a572be-e7f1-4526-b801-ba8bafa9a45a tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1451 microversion: 2.1 time: 0.152371 Oct 12 02:45:06.178669 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 583/1167] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:45:06 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1451 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:45:07.949301 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-702fe1c0-035b-404f-8a8e-c143aaf0b708 req-7115d798-f814-4ac3-861f-4b47a4b65d3e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "269ae487-b0f5-4513-a5a9-baa48cc9883a", "tag": "6a932fdd-a6eb-4530-952f-21adba6785ea"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:45:07.959684 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-702fe1c0-035b-404f-8a8e-c143aaf0b708 req-7115d798-f814-4ac3-861f-4b47a4b65d3e service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:45:07.960254 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-702fe1c0-035b-404f-8a8e-c143aaf0b708 req-7115d798-f814-4ac3-861f-4b47a4b65d3e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:45:07.960773 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-702fe1c0-035b-404f-8a8e-c143aaf0b708 req-7115d798-f814-4ac3-861f-4b47a4b65d3e service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:45:07.971834 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-702fe1c0-035b-404f-8a8e-c143aaf0b708 req-7115d798-f814-4ac3-861f-4b47a4b65d3e service nova] Creating event network-changed:6a932fdd-a6eb-4530-952f-21adba6785ea for instance 269ae487-b0f5-4513-a5a9-baa48cc9883a on hv-cinder-84914 Oct 12 02:45:07.977042 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-702fe1c0-035b-404f-8a8e-c143aaf0b708 req-7115d798-f814-4ac3-861f-4b47a4b65d3e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031301 Oct 12 02:45:07.977847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 585/1168] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:45:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:46:40.969815 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=26 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:46:40.970561 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=62 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:46:40.971527 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:46:40.971678 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Wed Oct 12 02:46:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 12 02:46:40.978276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1170658220", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-789738130", "security_groups": [{"name": "tempest-secgroup-smoke-650875271"}], "block_device_mapping_v2": [{"uuid": "427d6e31-d9b4-403b-9859-9f1898133c70", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c8f4d944-2283-4081-9e18-98805e4df4db"}]}} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:46:41.092063 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.network.neutron [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] validate_networks() for [('c8f4d944-2283-4081-9e18-98805e4df4db', None, None, None, None, None)] {{(pid=118865) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 12 02:46:41.495449 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Flavor limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 12 02:46:41.495874 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Image limits 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 12 02:46:41.496379 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Flavor pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 12 02:46:41.496821 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Image pref 0:0:0 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 12 02:46:41.497232 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.virt.hardware [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118865) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 12 02:46:41.505904 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:41.506387 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:41.506822 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:41.507545 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:41.508001 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:41.508425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:41.525297 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 12 02:46:41.529275 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.quota [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Getting quotas for user e184afaa0abd4e8eb1bc641167dc0b9b and project 0420b1fe8ff5451dba85de3001a3932f. Resources: {'ram', 'cores', 'instances'} {{(pid=118865) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 12 02:46:41.585046 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Going to run 1 instances... {{(pid=118865) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 12 02:46:41.587130 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='427d6e31-d9b4-403b-9859-9f1898133c70',volume_size=None,volume_type=None)] {{(pid=118865) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 12 02:46:41.714580 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-fd17b696-16dd-4c9c-b2ee-b385e3718364 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.746476 Oct 12 02:46:41.715293 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 584/1169] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 12 02:46:40 2022] POST /compute/v2.1/servers => generated 405 bytes in 748 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 12 02:46:41.724953 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:46:41.725748 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:46:41.732018 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-fbd1d6a4-b282-4647-bd7b-4de63dc52e8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:41.732747 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-fbd1d6a4-b282-4647-bd7b-4de63dc52e8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:41.794209 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-fbd1d6a4-b282-4647-bd7b-4de63dc52e8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:41.801132 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-fbd1d6a4-b282-4647-bd7b-4de63dc52e8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1094 microversion: 2.1 time: 0.078338 Oct 12 02:46:41.801699 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 586/1170] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:41 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1094 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:42.824091 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:42.826689 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:42.834592 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:42.835597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:42.835597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:42.836217 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:42.836644 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:42.837104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:42.908724 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:42.916463 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-4157a951-4b9d-4be2-9681-1e22ca7506f1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1210 microversion: 2.1 time: 0.096939 Oct 12 02:46:42.917059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 585/1171] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:42 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1210 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:43.939455 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:43.941233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:43.949016 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:43.949497 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:43.949925 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:43.950615 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:43.951066 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:43.951483 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:44.087679 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:44.096153 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f8b47998-4a7b-4b76-9f87-3bf5cb27f2b9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.160506 Oct 12 02:46:44.097998 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 587/1172] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:43 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:44.236134 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-447cc8a2-d285-47e2-9203-a6b81eed42d9 req-3ef8d261-99e6-48b4-9192-071d38c7a418 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7d50f8e-f68c-430e-9e72-ab304648b8c7", "tag": "3f5b0fe8-008f-45a2-9957-cc7906deea91"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:46:44.247240 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-447cc8a2-d285-47e2-9203-a6b81eed42d9 req-3ef8d261-99e6-48b4-9192-071d38c7a418 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:44.247240 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-447cc8a2-d285-47e2-9203-a6b81eed42d9 req-3ef8d261-99e6-48b4-9192-071d38c7a418 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:44.247618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-447cc8a2-d285-47e2-9203-a6b81eed42d9 req-3ef8d261-99e6-48b4-9192-071d38c7a418 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:44.259491 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-447cc8a2-d285-47e2-9203-a6b81eed42d9 req-3ef8d261-99e6-48b4-9192-071d38c7a418 service nova] Creating event network-changed:3f5b0fe8-008f-45a2-9957-cc7906deea91 for instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 on hv-cinder-84914 Oct 12 02:46:44.265364 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-447cc8a2-d285-47e2-9203-a6b81eed42d9 req-3ef8d261-99e6-48b4-9192-071d38c7a418 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096258 Oct 12 02:46:44.265968 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 586/1173] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:46:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 12 02:46:45.122403 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:45.123094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:45.129390 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:45.129877 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:45.130309 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:45.131000 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:45.131453 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:45.131878 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:45.261749 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:45.269560 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-47364f38-a7fb-4181-8f5c-67c2cefcbc71 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.151037 Oct 12 02:46:45.270119 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 588/1174] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:45 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:46.292322 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:46.293047 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:46.299149 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:46.299627 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:46.300059 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:46.300765 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:46.301220 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:46.301640 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:46.433160 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:46.440660 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-db779fc5-eb37-4e2c-a126-9dace9b55cb2 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.152992 Oct 12 02:46:46.441251 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 587/1175] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:46 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:47.463955 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:47.464565 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:47.470992 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:47.471476 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:47.471911 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:47.472607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:47.473084 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:47.473506 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:47.605374 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:47.613021 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-376991e2-aed7-474e-a4d8-a58be7e41e42 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.152872 Oct 12 02:46:47.613584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 589/1176] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:47 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:48.635766 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:48.636369 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:48.642293 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:48.642765 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:48.643195 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:48.643866 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:48.644310 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:48.644743 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:48.771219 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:48.778740 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-23c65416-0d87-45e6-8214-aff533d01b7b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.146981 Oct 12 02:46:48.779299 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 588/1177] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:48 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:49.801974 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:49.802575 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:49.809037 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:49.809508 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:49.809958 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:49.810627 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:49.811075 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:49.811489 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:49.937240 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:49.944861 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-69b1b636-c08d-415f-86ad-c37f6b4223ad tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.146930 Oct 12 02:46:49.945418 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 590/1178] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:49 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:46:50.966713 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:50.967316 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:50.973674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:50.974156 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:50.974588 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:50.975858 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:50.976321 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:50.976762 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:51.108796 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:51.116516 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-6d2e5a9d-b4c9-4feb-b05b-b33ac57f0d8b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.153675 Oct 12 02:46:51.117113 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 589/1179] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:50 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:52.139363 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:52.139972 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:52.142319 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell0:SELECT=2 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:46:52.148340 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:52.148860 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:52.149296 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:52.149987 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:52.150445 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:52.150868 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:52.280623 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:52.288262 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-d38ebd69-6b48-496e-9d2d-6cae71fa5738 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.152783 Oct 12 02:46:52.288847 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 591/1180] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:52 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:53.310094 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:53.310702 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:53.312662 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=2 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:46:53.313345 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell0:SELECT=6 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:46:53.319849 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:53.320330 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:53.320790 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:53.321506 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:53.321958 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:53.322373 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:53.451029 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:53.458813 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2a5fedb1-2c20-4ecb-8d9e-a07dc93bed96 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.152558 Oct 12 02:46:53.459383 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 590/1181] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:53 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:54.482954 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:54.483644 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:54.489919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:54.490398 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:54.490855 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:54.492113 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:54.492557 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:54.493001 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:54.524454 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [req-b45c41b7-e603-40da-8907-75d56c7cf86d req-d6df8202-9c8e-4889-a6af-7d59e9e0eaa2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d7d50f8e-f68c-430e-9e72-ab304648b8c7", "name": "network-vif-plugged", "status": "completed", "tag": "3f5b0fe8-008f-45a2-9957-cc7906deea91"}]} {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:46:54.537081 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b45c41b7-e603-40da-8907-75d56c7cf86d req-d6df8202-9c8e-4889-a6af-7d59e9e0eaa2 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:54.537613 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b45c41b7-e603-40da-8907-75d56c7cf86d req-d6df8202-9c8e-4889-a6af-7d59e9e0eaa2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:54.538053 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [req-b45c41b7-e603-40da-8907-75d56c7cf86d req-d6df8202-9c8e-4889-a6af-7d59e9e0eaa2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:54.550785 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.compute.server_external_events [req-b45c41b7-e603-40da-8907-75d56c7cf86d req-d6df8202-9c8e-4889-a6af-7d59e9e0eaa2 service nova] Creating event network-vif-plugged:3f5b0fe8-008f-45a2-9957-cc7906deea91 for instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 on hv-cinder-84914 Oct 12 02:46:54.558532 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [req-b45c41b7-e603-40da-8907-75d56c7cf86d req-d6df8202-9c8e-4889-a6af-7d59e9e0eaa2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037905 Oct 12 02:46:54.559460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 591/1182] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:46:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:46:54.630599 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:54.638331 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-8ca9af13-53c6-4e3b-9116-d417725cc952 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.159548 Oct 12 02:46:54.638887 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 592/1183] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:54 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:55.662115 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:55.662810 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:55.669700 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:55.670173 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:55.670597 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:55.671276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:55.671718 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:55.672129 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:55.803319 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:55.811566 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-eb4af497-8634-40fb-89bd-80a0a3b3fe2b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1265 microversion: 2.1 time: 0.153814 Oct 12 02:46:55.822096 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 592/1184] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:55 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:56.835405 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:56.836107 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:56.842426 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:56.842912 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:56.843346 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:56.844031 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:56.844494 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:56.844935 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:56.975251 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:56.983739 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f3704d4e-c865-4ce8-811a-cf12be81af3b tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1452 microversion: 2.1 time: 0.152196 Oct 12 02:46:56.984297 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 593/1185] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:56 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1452 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:57.005664 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:46:57.006276 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:46:57.012572 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:57.013087 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:57.013526 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:57.014216 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:57.014672 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:57.015098 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:57.144969 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:46:57.153090 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-72e6ede0-4c87-42f2-becc-7e1c7e3ed8b1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1452 microversion: 2.1 time: 0.151314 Oct 12 02:46:57.153643 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 593/1186] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:46:57 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1452 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:46:58.691189 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-57a3e563-6b77-4364-a7dd-b5e71f905e7e req-4c6b5a75-b191-4013-b1af-87bd605f7240 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7d50f8e-f68c-430e-9e72-ab304648b8c7", "tag": "3f5b0fe8-008f-45a2-9957-cc7906deea91"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:46:58.700789 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-57a3e563-6b77-4364-a7dd-b5e71f905e7e req-4c6b5a75-b191-4013-b1af-87bd605f7240 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:46:58.701274 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-57a3e563-6b77-4364-a7dd-b5e71f905e7e req-4c6b5a75-b191-4013-b1af-87bd605f7240 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:46:58.701727 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-57a3e563-6b77-4364-a7dd-b5e71f905e7e req-4c6b5a75-b191-4013-b1af-87bd605f7240 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:46:58.713668 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-57a3e563-6b77-4364-a7dd-b5e71f905e7e req-4c6b5a75-b191-4013-b1af-87bd605f7240 service nova] Creating event network-changed:3f5b0fe8-008f-45a2-9957-cc7906deea91 for instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 on hv-cinder-84914 Oct 12 02:46:58.718666 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-57a3e563-6b77-4364-a7dd-b5e71f905e7e req-4c6b5a75-b191-4013-b1af-87bd605f7240 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031097 Oct 12 02:46:58.719252 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 594/1187] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:46:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:48:30.974329 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:48:30.975617 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG dbcounter [-] [118865] Writing DB stats nova_cell1:SELECT=48 {{(pid=118865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:48:30.976682 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:48:30.977071 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Wed Oct 12 02:48:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 (ip 10.222.0.23) !!! Oct 12 02:48:30.979538 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_api:SELECT=20 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:48:30.981169 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG dbcounter [-] [118864] Writing DB stats nova_cell1:SELECT=44 {{(pid=118864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 02:48:30.982163 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:48:30.982728 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Wed Oct 12 02:48:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.23) !!! Oct 12 02:48:30.982939 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:30.983265 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:30.984063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 12 02:48:30.984193 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Wed Oct 12 02:48:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.23) !!! Oct 12 02:48:30.993799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-8568e4f5-ccc6-4c74-9917-fe70fdae309d req-d4a2ebc3-744a-4169-803b-cbc776f5ac2d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7d50f8e-f68c-430e-9e72-ab304648b8c7", "tag": "3f5b0fe8-008f-45a2-9957-cc7906deea91"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:48:30.995814 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:30.996509 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:30.997191 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:30.998116 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:30.998800 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:30.999728 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:31.007179 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-8568e4f5-ccc6-4c74-9917-fe70fdae309d req-d4a2ebc3-744a-4169-803b-cbc776f5ac2d service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:31.007658 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-8568e4f5-ccc6-4c74-9917-fe70fdae309d req-d4a2ebc3-744a-4169-803b-cbc776f5ac2d service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:31.008094 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-8568e4f5-ccc6-4c74-9917-fe70fdae309d req-d4a2ebc3-744a-4169-803b-cbc776f5ac2d service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:31.020210 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-8568e4f5-ccc6-4c74-9917-fe70fdae309d req-d4a2ebc3-744a-4169-803b-cbc776f5ac2d service nova] Creating event network-changed:3f5b0fe8-008f-45a2-9957-cc7906deea91 for instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 on hv-cinder-84914 Oct 12 02:48:31.025214 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-8568e4f5-ccc6-4c74-9917-fe70fdae309d req-d4a2ebc3-744a-4169-803b-cbc776f5ac2d service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047813 Oct 12 02:48:31.025826 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 595/1188] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:48:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:48:31.034192 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:48:31.134956 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-042fd48a-5620-40b3-96bb-b4a124383538 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "DELETE /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 204 len: 0 microversion: 2.1 time: 0.162249 Oct 12 02:48:31.138645 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 594/1189] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:48:30 2022] DELETE /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:48:31.145458 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:31.146099 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:31.152333 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:31.152837 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:31.153272 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:31.153958 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:31.154409 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:31.154827 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:31.299683 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:48:31.307447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-3abd685f-15e8-4e75-a360-15fcbee6eace tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1577 microversion: 2.1 time: 0.167276 Oct 12 02:48:31.308018 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 596/1190] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:31 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1577 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:48:32.331486 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:32.332184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:32.338476 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:32.338960 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:32.339393 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:32.340088 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:32.340537 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:32.340980 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:32.477816 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:48:32.486376 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d7659466-88a9-4401-b9f0-603a9a23c1eb tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1577 microversion: 2.1 time: 0.159161 Oct 12 02:48:32.486961 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 595/1191] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:32 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1577 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:48:33.509477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:33.510107 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:33.516115 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:33.516860 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:33.517620 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:33.518308 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:33.519055 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:33.519642 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:33.555051 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-d2ab8223-94db-4129-885f-43ae4d6d80ae req-4fa5a2f2-c81d-49e7-ab15-2d755385a5b2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d7d50f8e-f68c-430e-9e72-ab304648b8c7", "name": "network-vif-deleted", "tag": "3f5b0fe8-008f-45a2-9957-cc7906deea91"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:48:33.565423 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d2ab8223-94db-4129-885f-43ae4d6d80ae req-4fa5a2f2-c81d-49e7-ab15-2d755385a5b2 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:33.566063 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d2ab8223-94db-4129-885f-43ae4d6d80ae req-4fa5a2f2-c81d-49e7-ab15-2d755385a5b2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:33.566499 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d2ab8223-94db-4129-885f-43ae4d6d80ae req-4fa5a2f2-c81d-49e7-ab15-2d755385a5b2 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:33.577597 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-d2ab8223-94db-4129-885f-43ae4d6d80ae req-4fa5a2f2-c81d-49e7-ab15-2d755385a5b2 service nova] Creating event network-vif-deleted:3f5b0fe8-008f-45a2-9957-cc7906deea91 for instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 on hv-cinder-84914 Oct 12 02:48:33.582607 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-d2ab8223-94db-4129-885f-43ae4d6d80ae req-4fa5a2f2-c81d-49e7-ab15-2d755385a5b2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.115281 Oct 12 02:48:33.583201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 597/1192] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:48:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:48:33.583808 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:48:33.592041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-0770b79f-a1a1-4644-8008-1d101326f755 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 200 len: 1225 microversion: 2.1 time: 0.086517 Oct 12 02:48:33.592592 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 596/1193] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:33 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 12 02:48:34.615625 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:34.616328 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: d7d50f8e-f68c-430e-9e72-ab304648b8c7] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:34.622469 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:34.622953 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:34.623392 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:34.624085 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:34.624545 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:34.625002 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:34.638429 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] HTTP exception thrown: Instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 could not be found. Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Request method failure captured: Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 HTTP/1.1 Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRimE-Fp-G6xyxkrLZMxoeKfW_1MJtImcqjDq-YhjXhXXA2s1mtCDjmw2-NvvZt4N4kAzXZ9UzDGw9kJmdGQ6BNVqdDB8F3nY025Vf1v5qzOsh372asUi60wK4h3R4kxjVD_CelLyPtN84tvTp2q0cNN1wqrNK1o7M4dPpOkH-bY7kpo Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:48:34.640017 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_0420b1fe8ff5451dba85de3001a3932f", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: e184afaa0abd4e8eb1bc641167dc0b9b Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 could not be found. Oct 12 02:48:34.641806 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'd7d50f8e-f68c-430e-9e72-ab304648b8c7'} Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 could not be found. Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:48:34.643661 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 could not be found. Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 could not be found. Oct 12 02:48:34.645559 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:34.647027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Returning 404 to user: Instance d7d50f8e-f68c-430e-9e72-ab304648b8c7 could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:48:34.647027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-f649f34d-dcf7-42b5-9f16-9d3892595366 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7" status: 404 len: 111 microversion: 2.1 time: 0.030533 Oct 12 02:48:34.647027 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 598/1194] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:34 2022] GET /compute/v2.1/servers/d7d50f8e-f68c-430e-9e72-ab304648b8c7 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 12 02:48:37.674147 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:37.674666 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:37.675920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-05761eff-39fe-446a-8487-c0d1c7b3a233 req-f4510d3f-9c19-457e-a84f-a648306e7e65 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "269ae487-b0f5-4513-a5a9-baa48cc9883a", "tag": "6a932fdd-a6eb-4530-952f-21adba6785ea"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:48:37.681153 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:37.681631 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:37.682061 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:37.682742 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:37.683184 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:37.683901 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:37.686202 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-05761eff-39fe-446a-8487-c0d1c7b3a233 req-f4510d3f-9c19-457e-a84f-a648306e7e65 service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:37.686920 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-05761eff-39fe-446a-8487-c0d1c7b3a233 req-f4510d3f-9c19-457e-a84f-a648306e7e65 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:37.687347 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-05761eff-39fe-446a-8487-c0d1c7b3a233 req-f4510d3f-9c19-457e-a84f-a648306e7e65 service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:37.699895 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-05761eff-39fe-446a-8487-c0d1c7b3a233 req-f4510d3f-9c19-457e-a84f-a648306e7e65 service nova] Creating event network-changed:6a932fdd-a6eb-4530-952f-21adba6785ea for instance 269ae487-b0f5-4513-a5a9-baa48cc9883a on hv-cinder-84914 Oct 12 02:48:37.705942 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-05761eff-39fe-446a-8487-c0d1c7b3a233 req-f4510d3f-9c19-457e-a84f-a648306e7e65 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033731 Oct 12 02:48:37.706547 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 599/1195] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:48:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:48:37.710768 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Going to try to terminate instance {{(pid=118865) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 12 02:48:37.790519 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-dd51a1c9-5734-4d03-b5ef-7554e18d2b22 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "DELETE /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 204 len: 0 microversion: 2.1 time: 0.120383 Oct 12 02:48:37.791027 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 597/1196] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:48:37 2022] DELETE /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 12 02:48:37.800036 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:37.800759 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:37.807124 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:37.807870 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:37.808517 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:37.809449 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:37.810104 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:37.810799 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:37.945071 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.policy [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118864) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:48:37.952984 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-c691de70-c0b7-4032-a892-0509ad9f0ba1 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1576 microversion: 2.1 time: 0.157232 Oct 12 02:48:37.953473 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 600/1197] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:37 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1576 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:48:38.976397 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:38.977138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:38.983618 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:38.984089 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:38.984524 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:38.985244 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:38.985700 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:38.986674 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:39.115828 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:48:39.123787 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-606af045-777e-4b8c-833f-048d503294f9 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1576 microversion: 2.1 time: 0.151536 Oct 12 02:48:39.124349 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 598/1198] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:38 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1576 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:48:40.053919 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [req-d03390e3-5ad2-4010-a31b-f36e01016448 req-bb7be9dd-520b-4c89-b6f5-4bab495b607c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "269ae487-b0f5-4513-a5a9-baa48cc9883a", "name": "network-vif-deleted", "tag": "6a932fdd-a6eb-4530-952f-21adba6785ea"}]} {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 12 02:48:40.065474 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d03390e3-5ad2-4010-a31b-f36e01016448 req-bb7be9dd-520b-4c89-b6f5-4bab495b607c service nova] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:40.066021 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d03390e3-5ad2-4010-a31b-f36e01016448 req-bb7be9dd-520b-4c89-b6f5-4bab495b607c service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:40.066447 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [req-d03390e3-5ad2-4010-a31b-f36e01016448 req-bb7be9dd-520b-4c89-b6f5-4bab495b607c service nova] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:40.078109 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.compute.server_external_events [req-d03390e3-5ad2-4010-a31b-f36e01016448 req-bb7be9dd-520b-4c89-b6f5-4bab495b607c service nova] Creating event network-vif-deleted:6a932fdd-a6eb-4530-952f-21adba6785ea for instance 269ae487-b0f5-4513-a5a9-baa48cc9883a on hv-cinder-84914 Oct 12 02:48:40.083065 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [req-d03390e3-5ad2-4010-a31b-f36e01016448 req-bb7be9dd-520b-4c89-b6f5-4bab495b607c service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032681 Oct 12 02:48:40.083774 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 601/1199] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 12 02:48:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 12 02:48:40.146570 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:40.147175 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:40.153552 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:40.154031 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:40.154460 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:40.155138 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:40.155584 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:40.155999 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:40.217907 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.policy [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e184afaa0abd4e8eb1bc641167dc0b9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0420b1fe8ff5451dba85de3001a3932f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118865) authorize /opt/stack/nova/nova/policy.py:199}} Oct 12 02:48:40.225556 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-2ea019a6-dbfc-4755-aa92-03dfad932510 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 200 len: 1224 microversion: 2.1 time: 0.082826 Oct 12 02:48:40.226112 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 599/1200] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:40 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 1224 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 12 02:48:41.249758 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:41.250450 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: 269ae487-b0f5-4513-a5a9-baa48cc9883a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:41.257682 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:41.258154 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:41.258584 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:41.259258 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:41.259702 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:41.260146 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:41.273628 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] HTTP exception thrown: Instance 269ae487-b0f5-4513-a5a9-baa48cc9883a could not be found. Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Request method failure captured: Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a HTTP/1.1 Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRimE-Fp-G6xyxkrLZMxoeKfW_1MJtImcqjDq-YhjXhXXA2s1mtCDjmw2-NvvZt4N4kAzXZ9UzDGw9kJmdGQ6BNVqdDB8F3nY025Vf1v5qzOsh372asUi60wK4h3R4kxjVD_CelLyPtN84tvTp2q0cNN1wqrNK1o7M4dPpOkH-bY7kpo Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:48:41.275201 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_0420b1fe8ff5451dba85de3001a3932f", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: e184afaa0abd4e8eb1bc641167dc0b9b Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance 269ae487-b0f5-4513-a5a9-baa48cc9883a could not be found. Oct 12 02:48:41.277074 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': '269ae487-b0f5-4513-a5a9-baa48cc9883a'} Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 269ae487-b0f5-4513-a5a9-baa48cc9883a could not be found. Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:48:41.278233 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 269ae487-b0f5-4513-a5a9-baa48cc9883a could not be found. Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 269ae487-b0f5-4513-a5a9-baa48cc9883a could not be found. Oct 12 02:48:41.279938 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.281452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Returning 404 to user: Instance 269ae487-b0f5-4513-a5a9-baa48cc9883a could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:48:41.281452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-06ece301-4a8e-40cf-a69e-6e5fa814d072 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a" status: 404 len: 111 microversion: 2.1 time: 0.030978 Oct 12 02:48:41.281452 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 602/1201] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:41 2022] GET /compute/v2.1/servers/269ae487-b0f5-4513-a5a9-baa48cc9883a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:48:41.705470 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:41.705957 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.compute.api [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118865) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:41.712477 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:41.712989 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:41.713419 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:41.714104 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:41.714549 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:41.714967 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG oslo_concurrency.lockutils [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:41.727760 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.wsgi [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] HTTP exception thrown: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Request method failure captured: Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: request: DELETE /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a HTTP/1.1 Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept: application/json Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Accept-Encoding: identity Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Connection: close Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Length: 0 Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Content-Type: application/json Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Host: 10.222.0.23 Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: Openstack-System-Scope: None Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: User-Agent: python-urllib3/1.26.12 Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Auth-Token: gAAAAABjRimE-Fp-G6xyxkrLZMxoeKfW_1MJtImcqjDq-YhjXhXXA2s1mtCDjmw2-NvvZt4N4kAzXZ9UzDGw9kJmdGQ6BNVqdDB8F3nY025Vf1v5qzOsh372asUi60wK4h3R4kxjVD_CelLyPtN84tvTp2q0cNN1wqrNK1o7M4dPpOkH-bY7kpo Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Id: None Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Domain-Name: None Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Identity-Status: Confirmed Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Is-Admin-Project: True Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Id: default Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Domain-Name: Default Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Project-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Role: member,reader Oct 12 02:48:41.729434 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Roles: member,reader Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_0420b1fe8ff5451dba85de3001a3932f", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-Tenant-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Id: default Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Domain-Name: Default Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Id: e184afaa0abd4e8eb1bc641167dc0b9b Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: X-User-Name: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: method: > Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: exception: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.731426 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: action_args: {'id': 'b42a22ef-f44f-4e88-88ca-2ea755acca2a'} Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:48:41.732542 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.734231 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.736507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Returning 404 to user: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. {{(pid=118865) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:48:41.736507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-d5b360aa-c96f-4f18-8967-31d15bc7f172 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "DELETE /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 404 len: 111 microversion: 2.1 time: 0.029651 Oct 12 02:48:41.736507 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 600/1202] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 12 02:48:41 2022] DELETE /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:48:41.740081 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '>' {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:41.740859 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.compute.api [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] [instance: b42a22ef-f44f-4e88-88ca-2ea755acca2a] Fetching instance by UUID {{(pid=118864) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 12 02:48:41.746569 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:41.747052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:41.747481 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:41.748159 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Acquiring lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 12 02:48:41.748606 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 12 02:48:41.749052 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG oslo_concurrency.lockutils [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Lock "dbde3dce-57ec-471e-b3b0-530d5203c2f2" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118864) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 12 02:48:41.762595 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.wsgi [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] HTTP exception thrown: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Request method failure captured: Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: request: GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a HTTP/1.1 Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept: application/json Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Accept-Encoding: identity Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Connection: close Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Length: 0 Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Content-Type: application/json Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Host: 10.222.0.23 Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: Openstack-System-Scope: None Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: User-Agent: python-urllib3/1.26.12 Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Auth-Token: gAAAAABjRimE-Fp-G6xyxkrLZMxoeKfW_1MJtImcqjDq-YhjXhXXA2s1mtCDjmw2-NvvZt4N4kAzXZ9UzDGw9kJmdGQ6BNVqdDB8F3nY025Vf1v5qzOsh372asUi60wK4h3R4kxjVD_CelLyPtN84tvTp2q0cNN1wqrNK1o7M4dPpOkH-bY7kpo Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Id: None Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Domain-Name: None Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Identity-Status: Confirmed Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Is-Admin-Project: True Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Id: default Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Domain-Name: Default Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Project-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Role: member,reader Oct 12 02:48:41.764070 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Roles: member,reader Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/0420b1fe8ff5451dba85de3001a3932f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_0420b1fe8ff5451dba85de3001a3932f", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}] Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Id: 0420b1fe8ff5451dba85de3001a3932f Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-Tenant-Name: tempest-TestVolumeBootPattern-608330013 Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Id: default Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Domain-Name: Default Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Id: e184afaa0abd4e8eb1bc641167dc0b9b Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: X-User-Name: tempest-TestVolumeBootPattern-608330013-project Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: method: > Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: exception: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.765779 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: action_args: {'id': 'b42a22ef-f44f-4e88-88ca-2ea755acca2a'} Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: {{(pid=118864) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 12 02:48:41.766769 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. Oct 12 02:48:41.768144 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: ERROR nova.api.openstack.wsgi Oct 12 02:48:41.769522 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: DEBUG nova.api.openstack.wsgi [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Returning 404 to user: Instance b42a22ef-f44f-4e88-88ca-2ea755acca2a could not be found. {{(pid=118864) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 12 02:48:41.769522 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: INFO nova.api.openstack.requestlog [None req-e715012b-1565-4101-8adc-3173d64a7486 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a" status: 404 len: 111 microversion: 2.1 time: 0.029714 Oct 12 02:48:41.769522 dv-cinder-849143-14-iscsi devstack@n-api.service[118864]: [pid: 118864|app: 0|req: 603/1203] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 12 02:48:41 2022] GET /compute/v2.1/servers/b42a22ef-f44f-4e88-88ca-2ea755acca2a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 12 02:48:43.062425 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: DEBUG nova.api.openstack.wsgi [None req-e4c1e424-d39f-4c90-b04e-01c454987a13 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] Calling method '.version_select at 0x7f59d3455d30>' {{(pid=118865) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 12 02:48:43.086380 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: INFO nova.api.openstack.requestlog [None req-e4c1e424-d39f-4c90-b04e-01c454987a13 tempest-TestVolumeBootPattern-608330013 tempest-TestVolumeBootPattern-608330013-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-789738130" status: 202 len: 0 microversion: 2.1 time: 0.027799 Oct 12 02:48:43.087041 dv-cinder-849143-14-iscsi devstack@n-api.service[118865]: [pid: 118865|app: 0|req: 601/1204] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 12 02:48:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-789738130 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)